If your Windows version claims it won't run a .com file, it probably won't. Your options are to install an emulator - "dosbox" is popular:
http://www.dosbox.com/wiki/ReleasesThat should allow you to run your example, and all the other examples of .com files you'll find scattered around. This is about the only reason to try to run .com files - lots of examples!
The other option is to find source code suitable for a "first program" for Windows - either 32-bit or 64-bit "should" work on your system. 32-bit examples should be easier to find, but you might want to "bite the bullet" and learn Windows 64-bit programming, since that's probably where you'll end up eventually.
It's a tradeoff. There's very little point in "learning dos" these days. It's obsolete. But it offers a fairly gentle introduction to assembly language. 32-bit Windows is (IMO) a little harder to learn - that is, harder to understand what you're actually doing. There should be examples easily found... although I can't come up with any at the moment(!)... 32-bit Windows is going to be similarly "obsolete" some day, I suppose. 64-bit would be the most "modern" system to learn, but tutorials and examples are harder to find, and are apt to assume that you already know assembly language...
Okay, here's a 32-bit Windows example:
http://forum.nasm.us/index.php?topic=762.msg2396;topicseen#msg2396This claims to offer a "gentle introduction" to 64-bit assembly:
http://www.x86-64.org/documentation.html(... although it assumes that you know 32-bit Gas well...)
Here's another tutorial for 64-bit, with some example code:
http://www.vikaskumar.org/amd64/index.htm(... although the examples are for Linux - uses Yasm, not Nasm, but those should be "just the same")
I think if I were you, I'd install "dosbox" and see how it goes from there!
...
[Edit] well, I see you tried that. Your code isn't quite right for a dos .com file... Try:
org 100h ; where dos will load us
mov dx, msg
mov ah, 9
int 21h
ret
msg db "hello world$"
Best,
Frank