This is a topic that has occurred on quite a few sites. I had the same problem using Jeff's book. For several days (9 Aug 2010) I've been communicating with him about this and another problem. Here are a bit of background info and some explanations/solutions:
Jeff wrote the book using Ubuntu 8.10 on a 32-bit machine, back before 64-bit CPUs and operating systems were widespread. People with that setup shouldn't run into any problems. The problem arises when you try to assemble and link on a 64-bit machine. As others have noted, when using the commands Jeff wrote, the
-f elf nasm command line option will produce an object module with
32-bit code. By default, the linker wants to see a system default
64-bit object module. It will fail to link and will produce the error message you mention.
nasm and
ld are very versatile utilities. If you would like to see all the different output formats that Nasm can produce, run the following command at your terminal prompt:
nasm -hfMeanwhile, there are two fixes I've found that work for people using 64-bit machines. First, to produce a native 64-bit object module and executable, simply change
elf to
elf64 in Jeff's command, like so:
nasm -f elf64 -g -F stabs eatsyscall.asm -l eatsyscall.lst
ld -o eatsyscall eatsyscall.o
Note, you don't want to change the
ld command from Jeff's in the book. These two commands will assemble your source code into a
64-bit object module (
eatsyscall.o) and then link that module to create an executable file (
eatsyscall)
meant to be run on a
64-bit machine. Admittedly, this is probably the simplest way to develop executable programs on a 64-bit machine and OS. A disadvantage is that your hard-crafted program (the executable file), being a 64-bit program, won't run on 32-bit machines like many of your friends have. If that's of concern to you, you should probably consider:
A second option, assembling and linking to produce a
32-bit executable, I prefer and recommend for several reasons. Trivially, 32-bit object and executable files are significantly smaller than their 64-bit equivalents. That's not a big issue now that terabyte hard drives are so cheap and common. More important, the executable files generated will run just fine on
both 32-
and 64-bit machines. To go this route (hint, hint), don't change Jeff's
nasm command;
-f elf is equivalent to
-f elf32; i.e., his command already produces a
32-bit object module (which we've already learned). What needs changing in this case is the
linker command. You need to tell
ld you want a
32-bit executable instead of the default
64-bit format. You do that with the following two commands from your terminal:
nasm -f elf -g -F stabs eatsyscall.asm -l eatsyscall.ls
ld -o eatsyscall eatsyscall.o -melf_i386 
Hope this helps in understanding this problem and providing a couple of "solutions."
BTW, the other, more serious, problem with which Jeff is currently dealing, is the fact that he was blindsided by the recent release of Ubuntu 10.4. For reasons not completely known, the Insight debugger, which Jeff uses extensively throughout his book, has been dropped from the Ubuntu 10.4 package. Further, an Insight package is not to be found on any of the standard repositories that Synaptic knows about. Insight packages from other sources don't seem to work, possibly a result of version conflicts in the Ubuntu 10.4 kernel, Insight, GDB, Kdbg, and/or poor installation packaging . . . at this point we don't yet know!
Bottom line: if you want the best experience working with Duntemann's book, try using an earlier version of Ubuntu (8.10 would be best, perhaps on a separate, bootable partition?), which came packaged with Insight. If you can do that, and understand and heed the above info about 32/64-bit machines and OSs, Jeff's instructions should work fine. I suspect this has been a valuable learning experience for many of us.
