There’s three ways that I think about the apps on my iPad:
These are the ones that are built into the iPad, like Photos, and Mail, and Settings. These live in a folder to the left of the dock.
These are the apps that are critical to my workflow, and I spend the most amount of time in. Like Ulysses, Things, Fantastical, and Tweetbot. I need them to be front and center for me to use right away.
These apps are still very important to me, but not as important as my dock apps. Netflix, 1Password, and YouTube are some of the apps that I often open the folder to get to.
Also: I turn off the suggested and recent apps feature for the dock, to keep random or recent apps from popping up on the side of my second folder.
Also: I’m always evaluating the apps I have downloaded. If I don’t need the app anymore, I get rid of it. All the apps I no longer need are always in the second folder, and usually in the back of the folder, because I move the ones I often use to the front.
Also: I use a similar thought process for the iPhone 1, with the two folder structure.