Keep in mind that I am not familiar with 64-bit code! I think I see the problem - we can only use a 32-bit immediate(?). Possibly something like this?
section .bss
var resq 1
section .text
;...
; 16 relevant qwords presumed on stack
;...
mov ebx, 0Fh
pop rax
shr rax, 60
; and rax, rbx ; useless?
or [var], rax
pop rax
shl rbx, 4
shr rax, 56
and rax, rbx
or [var], rax
pop rax
shl rbx, 4
shr rax, 52
and rax, rbx
or [var], rax
pop rax
shl rbx, 4
shr rax, 48
and rax, rbx
or [var], rax
pop rax
shl rbx, 4
shr rax, 44
and rax, rbx
or [var], rax
pop rax
shl rbx, 4
shr rax, 40
and rax, rbx
or [var], rax
pop rax
shl rbx, 4
shr rax, 36
and rax, rbx
or [var], rax
; etc...
pop rax
shl rbx, 4
; shr rax, 0
and rax, rbx
or [var], rax
That assembles without complaint. I have no idea if it actually works. I have no idea if it's what you have in mind. I have no idea what it would be useful for.
Best,
Frank