Weights can be baked into silicone or programmed into hardware ala FPGA, but the context will always be dynamic.

High speed SRAM is where the $$$ is