Including structures and huge coff(ee) files

[nasm32]

If I include a huge include file with many structures, the size of the coff file increases to tremendous size of almost a Megabyte. Why does it do that? I've checked if the file is included more than once, it's not. And the include file has no "broken" structures (include file healthy).

The size of the include file is 1.2 MB (In which only about 15% of the contents is structures, all the remaining are equates, without exception)

[Rob Neff]

If I include a huge include file with many structures, the size of the coff file increases to tremendous size of almost a Megabyte. Why does it do that? I've checked if the file is included more than once, it's not. And the include file has no "broken" structures (include file healthy).

The size of the include file is 1.2 MB (In which only about 15% of the contents is structures, all the remaining are equates, without exception)

Structure definitions in and of themselves do not increase file size.  It's only when you create instances of those structures, in a data segment for example, that the compiler/assembler has to reserve space in the object file for all the structure members.  It's not good programming practice to define structure instances from within an include file - only the definition, or layout, of that structure.

[nasm32]

I've found that the equates is responsible for the tremendous size. I removed a big chunk of the equates and the object file size dropped by some 200 K, removed another big chunk of the equates and another big drop by 200+- K. That's odd behavior, equates shouldn't increase the size of the object file. Perhaps there is some fuss with the format of the include file that I can't detect with my eyes. I am suspicious, I downloaded it from a GNU page on sourceforge.

It's either a corrupt file, or some of the equates is mistaken by the preprocessor for a macro and creates huge amount of unwanted data. I refer to no of the equates in my source and no structures are used. (Bah, i'm dumping the entire include file and create my own from scratch)

[gammac]

If I include windows.inc I get ~300 KB sized object files, I hate this behaviour too.

[nasm32]

I used the word "format" in my previous post, what I meant to say was encoding. Perhaps the file was encoded differently than what nasm.exe expects to read (Pure Ascii vs Unicode vs a mix of pure text, html and what not)

The encoding might be bad. I opened the include file in notepad++ to see what encoding it detects, and it is pure ansi. I tried opening the file in a hex editor to see if the file contains "unwanted" characters, couldn't find anything. The include file has clean "pure text encoding".

If I create a new include file from scratch, the same problem persist with clean text files too. My next suspicion may be that the assembler does something differently based whether you use SEGMENT or SECTION in your source code which perhaps has consequences on how it interpretes include files. I'm not sure about that, can't tell.

The code I used to include the file looks exactly like this:

Code: [Select]

%include 'z:\nasm\include\test.inc'
%include 'z:\nasm\include\macros.asm'

global start
extern ExitProcess
extern Int2Msg

secton .data use32
  _caption db 'Temporary program',0

section .text use32

start:
  stdcall Int2Msg, 100, _caption
  invoke ExitProcess, 0

Ignore the Int2Msg, it's just to print numbers in message boxes, I use it to see what is in registers from time to time. That is not what causes any errors, if I remove it, the problem with the include file persists.

The parameters I use on the command line is:
This is what is inside the batch file I use to compile

Code: [Select]

@echo off
nasm -f win32 -Ox Temp.asm
golink /entry start Temp.obj z:\nasm\include\coff\Int2Msg.obj kernel32.dll user32.dll

I normally use make files, but I have a test folder where I have a template for "messing around with any code", so I use a batch file to compile that one. The batch file is clean, everything assembles and links well, the final executable is of "proportional size". But the coff file is widely out of proportions.

The macros.asm file is not what causes the problem either, it's just a few custom macros I made, I can put it in here:

Code: [Select]

; Direct call through embedded symbols

%macro stdcall 1-*
%if %0 > 1
      %rep %0-1
%rotate -1
        push %1
      %endrep
      %rotate -1
%endif
call %1
%endmacro

; Indirect call through imports (Use greedy macro parameters, last parameters gets lumped into one)

%macro invoke 1-2+
stdcall [%1],%2
%endmacro

; Indirect comcall [objptr, method, args]

%macro cominv 2-*
  %if %0 > 2
    %rep %0-2 ; arg0 is always amount of arguments passed to macro (skip object and method [-2])
    %rotate -1 ; rotate last argument into %1
      push %1 ; push in reverse order
    %endrep
    %rotate -2 ; get the order back to normal again so that original arg1 is arg1
  %endif
  mov eax,[%1]
  push eax
  mov eax,[eax]
  call dword [eax+%2]
%endmacro

; Push many registers in straight order

%macro pushm 1-*
    %rep %0
      push %1
      %rotate 1
    %endrep
%endmacro

; Pop many registers in straight order

%macro popm 1-*
    %rep %0
      pop %1
      %rotate 1
    %endrep
%endmacro

The test.inc can be downloaded here and I'm using nasm version 2.11.05 (compiled on May 21, 2014)

nasm.exe hashes:
MD5:

6407f2f8527b6a7676498198a0dd9114

SHA-1:

5b4fa8ded7ae7b82234b70ac4396c8777fbd5189

[nasm32]

I have been digging a little bit into the very large coff files. nasm stores every equ and every structure element as static symbols into the coff file, even if you don't refer to them.

There must be something inside nasm that "automatically" refers them and then includes them by itself by mistake. This "fault" only happens when you specify win32 coff files, not with OMF objects.

I tried to use the ABSOLUTE directive just above the %include directive, to try to redirect the equates and structure reserves into "hypothetical space" but it didn't work, it had no effect.

I performed a new test. I removed all include files, stripped down the source to the bone skeleton with only ExitProcess executing. If you declare one EQU in the source, it adds 18 bytes to the COFF object file. To each equate you add to your source, it adds another 18 bytes to the object file, even if you never refer them.

I created a macro that repeated itself to create 10000 equates, and it added 10000 * 18 bytes to the object file. (Again, it has no effect in OMF objects, only win32 COFF objects)

Here is the macro that generates the equates:

Code: [Select]

global start
extern ExitProcess

%assign x 0
%rep 10000
  L%[x] EQU x
  %assign x x+1
%endrep

section .data
  _caption db 'Temporary program',0

section .text

start:
  push 0
  call [ExitProcess]

size of COFF object file after assembling:

180 350 bytes

size of OMF object file after assembling:

174 bytes

[nasm32]

Can we expect to see a fix to this in the near future?  (It's not particulary fun to write 500 MB of data every time I run make) I'm not fond of using OMF objects, they are obsolete and unusable for any larger projects. COFF objects are superior on the windows platform. There are many good assemblers out there that I use and they work properly with ms coff objects, but I kind of like nasm's preprocessor and want to continue using the assembler for that reason alone. A fix would be highly appreciative.