> I guess it looks like it's down since it's a slice but the effect is an inward sphere?

Yes, gravity is a vector field: every point in space near a heavy body has a vector pointed at the center of the body with a magnitude of the field strength. https://en.wikipedia.org/wiki/Gravitational_field

Whenever someone uses fabric sheet analogy, they need to shout that the X-Y of the sheet is a 2 dimensional analog of X-Y-Z space, and the Z direction of the sheet is the field magnitude, with the slope indicating direction.

All models are wrong, but some are useful (for understanding).