They had to work the whole day, dawn to dusk, just to survive.

I don’t think that’s true. At least the stuff I’ve read seems to show that about ~40 hours a week was pretty normal.

Don’t get me wrong, there’s good reasons to not want to go back to hunter-gatherer societies (the linked article goes into that), but raw hours worked isn’t one of them.