What happens to us when we die?

We go back to where we came from...

But I don't remember where I came from...

Neither do I...

So how will we know how to go back to where we came from if we don't remember where we came from??