Every networking project is a very complex system. Of course there are some main rules, but they are very i mean very general. If we consider project a conveyor, by which some data flows, we will see some places, where the flow is fast and some places where the flow is slow. Slow places are called bottlenecks.
The aim of every tuning engineer is determining these bottlenecks and removing them by rearranging system, rewriting code and changing machines or just reducing their effect using some tricks like load balancing etc.

First thing to do - try write a schema of your work on paper. Try to think, which bottlenecks you can get.

NB: It's much easier to do when developing project rather when running it in production.

Various bottlenecks require various methods. E. g. apache latency can be improved by tuning httpd. However there were times when a broken patch-cord was rushing all the work ;) You never know, where and when jam will occur. When you work with production-state system you'll get not only recent bugs and bottlenecks, but those which could be removed on a paper-stage. So

Preventing troubles is much easier and cheaper, than solving them.