Should I disable Smarty cache when using Page Cache Ultimate?

No, it is highly recommended to keep the Smarty cache enabled. This will speed up pages that are not cached and also pages where the cache is not yet generated.