I'd go with:
precision equ 30
; and later...
mov cx, precision
"%define precision 30" should also work, and do about the same thing (it's just a text substitution, not a true "constant"). If you're going to make it a variable, "db" would be incorrect. I see you've got it as "dw" in your code, which should also work, but as:
mov cx, [precision]
Nasm knows how big cx is, so saying "byte" isn't going to work. You could say "word", but you don't need to. If you really really wanted to make it "db",
movzx cx, byte [precision]
... but there's no advantage to it. If you might want to let the user select the precision at run-time, a variable would be the way to do it. If you can decide the precision at assemble-time, I'd go with "equ".
I haven't even looked at your code, aside from this one issue. What kind of problem have you had with it?
Best,
Frank