… for Assembly Language Programming
To get started with assembly language on your x86 box, here is a set of open source (free) toolchain elements that play well together when host and target platforms are an x86 PC running a Windows (or Linux) operating system…
An Open Source Toolset for Assembly Language Programming
In the Windows case, it is essential to separate discussions of 16-bit assembly language programming using MS-DOS facilities from 32-bit assembly language programming using the Win32 API, since the two involve quite different issues.
A Sandbox for a Gentle Introduction
Before I list the hardcore, bare metal tools, let me mention a gentler option for playing with 32-bit Intel x86 assembly language without having to extend your assembly code with Windows or C utility functions. The gentler option is GNU’s FreeBasic. This has a clean inline assembly syntax that gives you clear access to everything that you could do in assembly, while at the same time providing you with the convenience that Basic offers, in its characteristically basic manner.
What does this mean? It means that you can draft up your assembly within the pleasant FreeBasic harness or sandbox. Then when you have everything debugged and working the way you want, you can simply copy and paste the inline assembly into an .asm, compile with NASM, link with gcc and off you go with tested code. Nice. This can take some of the pain away when first starting out.
Now, for the hardcore, peel back my fingernails, I wanna code in assembly and eat nails for breakfast using only the bare bones tools, please ignore these gentler paragraphs and carry on. Hooah.
The Bare Metal Toolset
16-bit x86 assembly language programming using MS-DOS facilities
- ms-debug – interesting to get a feel for machine language (hex entry)
- SST – command line debugger for 16/32-bit object code
- NASM (16-bit assembler using Intel syntax, object code format switches: -f bin, -f bin with exebin macro, or -f obj)
- VAL (16-bit linker that takes NASM -f obj and produces omf exes)
32-bit x86 assembly language programming using win32 facilities
- PEBrowse – visual debugger for 32-bit object code
- NASMW (32-bit assembler using Intel syntax, object code format switch: -f win32)
- gcc (MinGW) – C compiler and 32-bit linker taking NASM -f win32 objects
- ld (MinGW) – 32-bit linker
- gas (MinGW) – 32-bit assembler using AT&T x86 syntax
- Microsoft link – 32-bit linker taking NASM -f win32 objects
- tcc – Tiny C Compiler, 32-bit linker taking NASM -f elf objects, 32-bit assembler using AT&T syntax
32-bit Linux assembly language programming
- flat file format NASM -f bin
- elf format NASM -f elf
Mixed language programming
- NASM & tcc for Assembly extended with C
- tcc for C with embedded Assembly (Intel syntax)
- NASM & gcc for Assembly extended with C
- gcc for C with embedded Assembly (AT&T syntax)
- vcc for C with embedded Assembly (Intel syntax)
Disassemblers & Debuggers
- ms-debug – good to start with, but only for training experience
- NASM Disassembler – good to get training experience with hex editor and layout of 16-bit DOS programs.
- PEBrowse – modern 32-bit visual debugger
This assembler assembles Intel syntax x86 assembly language into one of a variety of object file formats that can be linked into an executable. NASM assembles source code into 16-bit flat binary (.bin, .com), 16-bit OMF obj (for linking into an OMF executable for MS-DOS with the VAL linker), 32-bit ELF obj (for linking, using tcc, into a PE-i386 executable for modern Windows), and 32-bit win32-coff obj (for linking, using gcc, into a PE-i386 executable for modern Windows).
- this compiler compiles into straightforward assembly language with straightforward use of procedure calls. Small program (10% of mingw’s gcc)
- ALSO an x86 assembler (AT&T syntax, gas style) (with .s files)
- Can handle GNU inline assembly with asm keyword
This compiler can be made to compile into straightforward assembly to compare with tcc. Use the switches: gcc -fno-exceptions -s -O0 -Os Tje default compilation is the most general and so understandably compiles into convoluted assembly language with spaghetti like flow through procedure calls.
The standardization, flexibility, and thorough documentation of gcc then makes it more desirable than tcc as a toolchain for bare essential programming and automatic code generation.
Both of these compilers use procedures from msvcrt.dll. These call other functions in msvcrt.dll, which use functions in kernel32.dll, which in turn use functions in ntdll.dll
Note: it isn’t useful to count TOTAL instructions in a disassembled exe (including kernel32.dll and ntdll.dll). It’s the cleanness of the application code’s assembly. The TOTAL instructions can be very large since it involves OS required code and OS provided functionality.
linkers (tcc, gcc, val, link)
– the first two linkers allow the creation of 32-bit PE-i386 format executables for modern windows. The Val linker creates only 16-bit MS-OMF format executables for MS-DOS. The two modern linkers are even more useful because they allow the mixing of code: the extension of assembly with C functions and the embedding of assembly functions into C.
disassemblers and debuggers (ms-debug, SST, NASM, PEBrowse)
Preparing a 16-bit MS-DOS assembly language program
For 16-bit MS-DOS assembly language programs, you are free to take advantage of all DOS facilities. This is done by putting DOS function codes into the ah register, setting up the parameters that the function needs, and then activating the DOS interrupt 0x21.
1. Machine Language
If you will be entering the program into debug using machine language, you need the hex opcodes, and you will proceed using the e100 command followed by spaces. You will have to have mapped out the memory and pointer referencing yourself by hand. (Debug needs to start program execution at 0100h.)
2. ms-debug Assembler for flat binary (COM)
If you will be entering the program into debug’s assembler, you would assemble beginning at 0100h and do it in two passes, first laying out the program and putting a place holder value for the pointer, then later filling the pointer’s value in when you have laid down the data to which it points and know where this resides. (Debug needs to start program execution at 0100h.)
3. NASM Assembler for flat binary (COM)
If you are entering the program in an assembly language and if you will be assembling it with NASM to obtain a binary targeted at the debug platform, then you will need to add the ORG 0100 instruction at the start of the program so that debug interprets relative references relative to the ORG. (Debug needs to start program execution at 0100h.) Otherwise, the rest is just the actual assembly code. Very easy. No setup overhead except for the ORG 0100 line at the start.
4. NASM Assembler for OMF EXE using exebin macro and some setup code
If you are entering the program in an assembly language and if you want an EXE for x86 and are assembling it with NASM including the exebin.mac macro, then you need to provide appropriate scaffolding to your assembly program for exebin to work. This simply means identifying sections (.text, .data), setting up the stack properly, and initializing the data segment register with the address held in the code segment register
5. NASM Assembler for OBJ object code; VAL linker to get OMF exe
If you are entering the program in an assembly language and if you want an OBJ file (OMF format) for x86 that will then be able to be linked into an EXE, and if you are compiling using NASM and linking using VAL, then you need to provide the appropriate segmentation and associated program scaffolding to your assembly program. This simply means identifying segments (code, data, stack), marking the ..start place, initializing the ds, ss, and sp registers, reserving the stack space, and setting the stacktop: address. Note: the three segments can be in any order, not necessarily in code, data, stack order.
Preparing a 32-bit Windows console assembly language program
For 32-bit Windows console assembly language programs, you CANNOT use DOS facilities. You have to use instead the Windows OS functions, and these are the Win32 API functions. Though the Win32 API functions are C style functions, so the way you’ll use them will also be the way you’ll be able to extend assembly language with C functions.
In assembly language for 32-bit x86 running Windows, there is no DOS and you don’t have access to software interrupts. How to access peripherals?
KEY INSIGHT: With Windows 32, the operating system is WINDOWS, and the I/O and other basic OS calls are no longer DOS interrupts. Instead, they are in Kernel.lib, etc. So in order to write assembly language programs and use a linker that creates PE-i386 32-bit executables, you CANNOT use DOS interrupts. That was for 16-bit x86 programming under DOS (or under the DOS emulator in Windows).
So as with the 16-bit model, you need need go through the operating system, in this case Windows, and use the standard Windows functions.
Now, even though the Windows operating system is written using C, the Windows standard library functions use the PASCAL calling convention (also called the standard calling convention). This means that, for any real world use of assembly language, you need to know and understand the fundamentals of standard calling for functions, that is, passing parameters on the stack and indicating the total number of bytes pushed so that the function can clean them up, i.e. pop them all of, clear its portion of the stack.
Summary: With stdcall (a Pascal calling convention) as opposed to _cdecl (the C convention), there are a fixed number of parameters, caller pushes them onto the stack in reverse order. The function being called knows exactly how many bytes to remove off the stack. (In NASM you could also use the higher level instructions ENTER and LEAVE.)
How does the function return values? Any return value is stored in EAX.
Where do you find these standard Window functions? They are in the include headers for both NASM and tcc, but that won’t help with anything but the names and the API interface. They are also in the Win32 Help file:
and also in an even better application:
which tells you which library they belong to so you can link the library in.
6. NASM Assembler for WIN32 object code; Microsoft link linker to get EXE
Many of the functions are in vcc’s kernel32.lib. You will link that in statically on the linking line. The library to be linked can come before the object code, but it is better style to put it after. Static libraries for Link are .lib Dynamic libraries are .dll (Windows convention)
7. NASM Assembler for ELF object code; TCC for linking to get EXE
The library to be linked must come AFTER the object code. Cannot link in vcc library — says invalid object code. Either use -llib and it finds it in -Ldir. Or give the full path for the library to be linked in. Static libraries for TCC are .a Dynamic libraries are .so (Unix convention)
Mixing Languages by Extension: Assembly extended with C
So, now, after writing an assembly language program for win32 and having to use functions from win32, there arise a number of practical matters that point toward extending Assembly with C:
- how is win32 a more efficient library than the C library
- for most people and the majority of basic functions, the C standard library is far more familiar than win32
- at the end of the day, using C instead of Windows at least gives you some greater generality and universality. Win32 will ONLY work on Windows operating systems
- C then becomes an interface for operating system functionality, abstracting away the different between DOS, Windows and other operating systems. The machine specific compiler deals with the specific mappings
- you can save your coding in assembly for the things for which you really care to have coded in assembly. For mathematicians like Knuth, this might be a core algorithm, a tight loop, an optimized graphics routine, etc.
Progressing this way makes sense. After all, Dennis Ritchie and Ken Thompson invented C as a sort of macro assembler, a way to automatically generate assembler without sacrificing any of the power and flexibility that assembler allows the programmer. That is also why, by the way, the C language is a fairly complex and sophisticated language to learn. Learning C is much easier after Assembly. Learning Assembly is easier after learning C. Whichever you learn first is going to be difficult.
Now, to use C functions, you have to use the C calling convention. In a nutshell, you push parameters onto the stack (in reverse order) and then YOU yourself clean up after yourself by following up the function call with a stack adjustment, typically add esp, byte 8, indicating the total number of bytes to adjust the stack pointer by. This number will be equivalent to the total size of all parameters pushed onto the stack.
Mixing Languages by Embedding: C with embedded Assembly
The MIXED LANGUAGE model of using another language and DROPPING INTO assembly from within that language (i.e. EMBEDDED assembly) is usually preferred over EXTENDING assembly with another language’s library calls or functions. Why? It is because of the sheer complexity of the assembly language toolchain and the comparative simplicity of staying within a higher level language’s toolchain and letting the compiler/interpreter handle the “absorption” of the embedded assembly.
You can use C and drop into Assembly.
You can use Basic and drop into Assembly. FreeBasic makes this very easy.
When an ASM block is opened, the registers ebx, esi, and edi are pushed to the stack, when the block is closed, these registers are popped back from the stack. This is because these registers are required to be preserved by most or all OS’s using the x86 CPU. You can therefore use these registers without explicitly preserving them yourself.”
Assembly Language Forensics: Signatures and Disassembly
When you’re working in assembly and its intricate toolchain, it helps to be able to look at a random intermediate file spit out by some link along the toolchain and identify where it came from and where it can be taken by some other element of the toolchain. It is also invaluable to be able to set a disassembler on the file and reverse out (unassemble) the object code and see a human readable source file.
To this end, let’s go over the main file formats, object and executable, their signatures, file formats, headers, and disassembly methods.
- PE-i386 – Portable Executable
- com – Common Object Format
- ELF – Embedded Linker Format
- win32coff – Windows 32 COFF
NASM has disassembler.
PEBrowse is much better.
NASM win32 object file – In Lister, starts with “L***”. No Netwide Assembler stamp in it.
Microsoft Link – In List, starts with “MZ”. No stamp in it. Takes NASM win32 object. Creates PE-i386 EXE.
NASM elf object file – In List, starts with “ELF”. Netwide Assembler stamp in it.
TCC linker – Takes NASM elf object. Creates PE-i386 EXE
When looking at a bare gcc created EXE (i.e. with -fno-exceptions -s -O0 -Os), notice the following (by comparing the list output with the EXE in a hex editor):
– the .rdata section begins at 0x0C00
– the .text section begins at 0x0400