This is a pretty good question, but there's a standard trick to answer it. You don't define "material" directly, you just say either:

1) Whatever physics studies is material.

2) Whatever makes something material, atoms, quarks, and those other subatomic particles are material. By "material", we just mean whatever the ultimate theory of those things says they have in common.

Neither of these responses guarantees that "material" is a meaningful concept. If panpsychism is true[0], then everything is somehow conscious, and you may no longer have a reasonable criterion for materiality.

For more context, you might try the Stanford encyclopedia article on physicalism: https://plato.stanford.edu/entries/physicalism/.

[0] Sadly, there are very talented contemporary philosophers who believe in panpsychism.