It works less well when you have a question that doesn't have an obvious answer.
For fun, try 'How can I write to a Parquet file in Java without Hadoop?'. It will vacillate between giving answers that pull in hadoop dependencies, and nonsense that doesn't compile at all.
(here's a useful proper answer: https://stackoverflow.com/questions/59939309/read-local-parq...)
Complex questions that involve things that aren’t well documented are obviously going to be harder for an LLM to answer, particularly when the LLM does not have a capability to test its own code.
I briefly checked if the mentioned dependencies exist (they do). Too late over here to actually try the code.
Main.java:27:33
java: org.apache.parquet.io.OutputFile is abstract; cannot be instantiated
Main.java:29:17
java: cannot access org.apache.hadoop.fs.Path
class file for org.apache.hadoop.fs.Path not found