Well, what's happening to cause the segfault (I think) is that you're not finding either -1 (why would you expect that?) or a linefeed in the buffer, so you keep going until you hit invalid memory.
I'm not seeing either of your messages. This is printf's "dirty little secret". It doesn't print anything until the buffer is flushed! This is often not noticed, because it prints everything at the end... which your program doesn't reach because of the segfault! Easiest way to fix this is to add a linefeed at the end of each string (before the terminating zero).
;
; initialized data is put in the data segment here
;
%define newline 10
prompt1 db "Enter a number: ", newline, 0
testoutmsg db "test that step worked", newline, 0
That will flush the buffer (the i/o buffer, not yours), and printf will print something.
Dr. Carter's "read_char" reads just one character, and returns it in al. You put all of eax in your buffer. The upper parts of eax are "garbage" - zeros, in fact. You probably want:
mov [buffer], al
But since you've got the address of "buffer" in esi, and you increment esi...
mov [esi], al
would probably be better. Then, you can loop back and "read_char" another, until you find that linefeed you're looking for.
At present, your "ASCII_to_Hex" isn't used. I'm not sure where you intend to call it, or how you intend to use the result. If you've got 'a' in your buffer, and you put it in dl and call this function, it'll return 61h in dl. If you just print this, it'll produce 'a' again. If you expect to see "61", you'll need to print two characters. Maybe "print_int" will do what you want there? (although it'll print it in decimal, not hex) See how it goes...
As a minimum, I'd look for characters you're actually going to find in your buffer - 0, not -1, probably, or newline maybe (if you put it there). I think "print_string" expects a zero-terminated string, so you may want to add that - I don't think "read_char" will return it for you...
When posting code, if you put the word "code" in square brackets ('[]'), as if it were a memory reference in Nasm, and "/code" (in '[]'s) at the end, it'll put your code in "code tags", which makes it a little easier to read and a lot easier to "cut and paste". We like it if you use 'em!
See if you can get rid of the segfault first, and come back with more questions (if any
).
Best,
Frank