If you need better performance than a higher level language affords you, I would not recommend programming directly in asm. Instead, I would write a compiler. Raw asm is seductive since the start up cost is relatively low. You can get started in an afternoon. The trouble is that writing correct assembly is much harder than high level code. You have to hold in your head the register state at all times. You have to know if the function you are calling will clobber registers that you need after the call and manually save/restore them. This will slow down your velocity and the resulting code will be long and difficult to read. It will also rely on a lot of undocumented information that only resided in your head while writing and has since been evicted. Good luck debugging a program written in assembly that no one has looked at for two months.

On the other hand, if you write your own non optimizing compiler, you can avoid a lot of these problems by, for example, tracking what registers a function writes and ensuring they are saved before a call and restored after. Then you can actually get the raw performance of handwritten asm without the pitfalls (the resulting code would still he harder to read and maintain than equivalent high level code, but at least it would be tractable). Even better, you can write your own high level assembler that is actually portable to other architectures. For example, instead of directly modeling x86_64, your compiler can model a cpu with 16 general purpose registers and a set of instructions that map to x86_64 instructions. An arm port would be straightforward since arm also has 16 general purpose registers and you can model x86_64 instructions as one or more arm instructions (and you have extra registers for x86_64 instructions that must be modeled as multiple arm instructions). Or you could do the reverse and model 32 general purpose registers using arm instructions and use predefined memory slots as virtual registers on x86_64.