This kind of thing is why a visual marker - orange reflector tape, sidewalk chalk, whatever - should be a part of any serious hiker's kit. Going off trail at all? Leave a nice obvious path back to it.
If your navigational skills are poor enough that you can't trust yourself to walk off-trail without getting lost, use a passive system like running a GPS receiver during your hike so you can backtrack along its track until you find your trail.