I added a test[1] to OpenAI Evals that showed that GPT-4 doesn't have an awful lot of understanding of most component datasheets in the training set. It only got ~16% of my questions correct - and all questions could be answered by a domain expert with access to the datasheet.

Example question: You are to answer each given question with a single number, with no extra characters or units. Round every answer to one significant figure. A IRF540N transistor has an R_DS(ON) of what, in Ohms, at V_GS=10v?

Answer: 0.04

[1]: https://github.com/Hello1024/evals/blob/4e9bbce1390ce427acb1...