Every networking project is a very complex system. Of course there are some main rules, but they are very i mean very general. If we consider project a conveyor, by which some data flows, we will see some places, where the flow is fast and some places where the flow is slow. Slow places are called bottlenecks.
The aim of every tuning engineer is determining these bottlenecks and removing them by rearranging system, rewriting code and changing machines or just reducing their effect using some tricks like load balancing etc.

First thing to do - try write a schema of your work on paper. Try to think, which bottlenecks you can get.

NB: It's much easier to do when developing project rather when running it in production.

Various bottlenecks require various methods. E. g. apache latency can be improved by tuning httpd. However there were times when a broken patch-cord was rushing all the work ;) You never know, where and when jam will occur. When you work with production-state system you'll get not only recent bugs and bottlenecks, but those which could be removed on a paper-stage. So

Preventing troubles is much easier and cheaper, than solving them.

What are these notes for?
In my life I was working for many companies affiliated with unix projects. If the management of this company was good, there was time, when developers' team(mainly production engeneers) were stuck. The popularity of project was growing and servers weren't able to adequatly handle the load growth. Some of the firms were just buying faster servers, blades and so on. But if the project (and the firm) was young, it just haven't got enough investments to do so. And we were put the task how to work on servers we got.
5 years of hard work told me, that the extensive solution is often the wrong way. Optimising and tuning is a better and more interesting, than just spending money. Some of the tips and tricks i'll try to write here.