I used to hold the same opinion, that code should be expressive, so we shouldn't need comments. However, I've found this to be false for a number of reasons:
1) Proper comments allow you to explain the reasoning behind the code, not just what the code does.
Excellent examples of this sort of commenting include ZFS's code[1] and Linux's SLAB memory manager[2].
This is important because a lot of the time, there are minutae that are extremely important to get right, but it's not immediately obvious why you did it that way. For instance, maybe there is a special reason fine-grained locking was implemented a certain way, perhaps there is a reason this method of IPC was chosen, etc. A lot of thought goes into properly architectured code, and without comments, future contributors have no idea why this works.
System architecture details become pa
Read the rest
rticularly important when you delve into more difficult problems. For instance, I've seen the following snippet of VHDL code:
process(clock)
if(rising_edge(clock)) then
delayed_signal <= signal;
end if
end process
This piece of code delays the signal by one clock cycle, but a lot of time in poorly documented code, there's no explanation why! Maybe there's another signal that needed to be synchronized with this one, so you need another clock cycle. Maybe this signal needed to be re-synchronized into the clock domain and one DFF does the trick. Without appropriate commenting, you'll never know the true purpose of this code, so if you change something around it, there's a higher likelihood that everything will break.
2) Proper comments also explain what isn't there: the things that have failed, and why you probably shouldn't try them again.
The OP mentions that sometimes you spend a lot of time optimizing a block of code, only to find out that it doesn't come out better in the long run. Why isn't it commented? By commenting "oh, we tried this other topology, it didn't end up being right for this scenario because of reasons x, y, and z", you can save a lot of grief for somebody down the road.
You might say that the source history should show this, but how often do you look at previous commits when coding? I find that most people ignore the tree until it's time to merge, and even if previous attempts are documented, their reason for removal is often abbreviated in the commit message: "wasn't fast enough"/"didn't work as expected".
You can see the result of this on a large scale, if you take a look at the file-drawer effect[3]. If you dump a huge amount of time/money into projects, and then don't let other people know what happened because it didn't turn out well, people are likely to repeat the same mistakes you made.
3) Proper comments make it easier.
I'll admit it: I'm lazy. If I don't write the comments that specify points 1/2 immediately, as I'm refactoring the code, I'll lose my train of thought. Even if I manage to get away and dump the ideas in another file, they probably won't be as clear as if I wrote it right then and there, with the source code to reinforce my arguments.
The time to context-switch between writing code and writing documentation should be as small as possible, so I'm a big fan of systems like Haddock[4] which allow you to write code and documentation and parse-out the docs into HTML for later browsing.
Ultimately, the code and your comments serve as knowledge for the future, so don't let your time go to waste: document what you did, why you did it, and why maybe other things didn't didn't work.
[1] https://github.com/illumos/illumos-gate/blob/master/usr/src/...
[2] http://lxr.free-electrons.com/source/mm/slab.c
[3] http://skepdic.com/filedrawer.html
[4] http://www.haskell.org/haddock/
Example of great Haddock docs: http://hackage.haskell.org/packages/archive/snap/0.12.0/doc/...
edit: formatting