Employers Say Students Need AI Skills. What If Students Don't Want Them?
insidehighered.com
insidehighered.com
You're right in that AI is a tool that might be necessary to use, but just like with a heavy industry machine tool that can chop your arm off if you use it wrong, there need to be operator training requirements.
As a matter of fact, I'd extend this argument the other way around: AI might be deterministic enough per your argument, but that's not an interlock. An interlock for an AI would be some type of interaction limiting that prevents the worst garbage outputs. We already have some "AI interlocks" like that, in the form of censoring it from generating "socially inappropriate" output. Maybe someone can invent further "AI fuses", for example when the AI itself detects it's being asked to do complex math, and it is known that this AI can't in fact do math. (This could work on either input, internal state, or output. Also note most interlocks aren't "perfect".)
"They need to learn the material and to do their job, not search the Internet."
Same thing for AI, it's a crutch at best, not a way of doing things, and especially not something you need to focus on while teaching
That can be literary research, Internet research and yes, ultimately if will have to be how to query language model expert system to solve various problems.
You can copy and paste a solution from a book/SO/LLM or you can use it to understand, improve, research and solve various tasks.
Especially when the tools are so liable to change at zero notice. It's not the responsibility of the public education system to do product training for megacorps.
(People who disagree with this: let's see your syllabus for AI 101 with the guarantee that all its contents will still be valid in 5 years time?)
Does anybody think that the state of AI is sufficiently stable that things taught to high school students will be valid by the time they enter the workforce?
Is there even anything generalizable to teach? The big pitch for AI is that it's supposed to be able to understand natural language. Then there's a long tail of specific hacks that you insert into prompts to get particular results. Any of which could change tomorrow. It's like teaching SEO in school.
Now, they are in the customer facing documentation part of the business. Think manuals for all the myriad products we make, in all the many languages, and all the many regulatory systems. The job is exacting, precise, and must make sense to readers at the end of the day. Many manuals go through countless revisions and take months to release under all the requirements. Large parts of the job are pretty boring and automatable though.
So, they've been aggressively perusing AI tools for large parts of the boilerplate work and the formatting work. It turns out, that most of the hard parts of the job, the xml formatting and translation, are very easy to automate with AI. As in, what usually took them months of manual editing to preform now takes them seconds, with a day or two of spot checking the language. Regulatory requirements review still take just as long.
Here is where all the nerds here on HN can't believe that you're not using LaTeX or something.'How could it have taken them months?!'
They're all English majors above 50 years old. They have never coded anything in their lives. All this work was previously being done in MS Word. They at least had some sort of central file repo beforehand, but now they all use git and managed to stand up their own servers. I'm sure all of this is being done poorly, but it's being done in the first place. The alternative was much worse.
It's not just that the AI is managing to help them make xml skeletons in the blink of an eye, it's that AI told them and taught them about xml at all. It taught them about git too, and I'm sure a lot of other things they never knew about. AI is a really patient teacher.
So, the thing is that there is a lot of internal pushback against AI. From what I was told, there is about half the team that refuses to touch AI. Thinks it's the devil, that they can always do it better, do it faster, etc.
So far, the company line is that they are right and that you can elect to use AI or not. But that boss I was talking to said that the internal vibe of the higherups is that in about 2 years, you're going to be forced to use it or let go. Their internal numbers (no idea what those are, this is all third-hand info now, believe me if you want) are pointing out that using AI has just so much better results that it's going to be a job requirement going forward.
So, as one piece of anecdata, doing at least a few jobs is going to be, very soon, chatting with a bot too.
I think the main difference is that AI makes it more accessible. Even with previous no-code or RPA solutions, you'd frankly need a programmer to build and maintain them despite what promises the salespeople might've told you. AI changes that, at least in my experience. A lot of our automated processes are now being made by our more digitally inclined workers, and I think you're absolutely spot on that it'll be something that is required of anyone going forward. Most of these are minor things, and if they need to scale to company wide tools they're probably still going to need an actual programmer, but a lot of them are good enough to be shared within departments.
Of course people feeding sensitive data into LLM solutions is an issue we also have to deal with.
Of course, the solution to that is to have an AI that scans your email for problematic language.
https://www.bbc.com/travel/article/20240222-air-canada-chatb...
You are potentially liable for the contents of your work emails. This includes if you put a stupid disclaimer at the bottom. People have gone to jail over the contents of chat. It's one thing to take notes on a criminal conspiracy and another thing to have the AI make one up and drop you in it because you didn't read the email it sent.
(I'm not entirely serious, but also not entirely joking. Looking for these kinds of red flags is probably more within the ability of contemporary systems than reliably writing prose.)
You need to start from high school, the AP classes need to be revamped. Currently they are focused on purely frequentist statistics. Frequentist statistics is great for most empirical sciences like biology. The formulas are mostly plug and play and even pure life science people with no mathematical talent can wield them without trouble. The problem is that they are very far from statistical learning.
Here's the current AP stats curriculum, it is meant to be equivalent to Stat 1.
https://library.fiveable.me/ap-stats
If you want to develop a strong foundation for ML, Unit 6, 7, and 8 ought to be thrown out entirely. The level they are taught at doesn't really teach anything more than plugging formulas. Unit 4.5 (Conditional Probability) and Unit 5 (Sampling) need to be further developed to cover the Bayesian theories, perhaps a segue into graphical models and Markov chains. Generative ML for example interprets likelihood as an information generator (since in the Bayesian formula, it is roughly the "inverse" of conditional probability), unfortunately most stats classes outside of physics and high level ML theory will never mention this. Heck most classically trained statisticians won't ever encounter this idea. But it is the bread and butter of generative AI. Having a vague idea of KL-divergence and what Metropolis-Hastings is coming out of high school is infinitely more useful for their career in ML than knowing how to fiddle with a p-value. You can teach most of these concept without calculus if you simplify some things and replace integrals with their discrete summation versions. Rejection sampling for example is very easy to teach. The Common Core needs a revamp, and perhaps it's time to shift away from the historical focus on calculus/pre-calc as the central tenet of pre-college mathematical teaching.
Believe me, most students of the common core don't know or care about the difference between Bayesian and frequentist statistics. Are you only interested in helping the straight-A students that have better chances getting into Harvard? Did you read the article?
The point is to build intuition, those take time and unless the student is particularly mathematically inclined, you can't rush it. That's why you have to start in high school. Right now, the focus of pre calc is to induce them into calculus through slopes, limits, and derivatives. But a good chunk of high level statistics don't need that to build intuition. For example, we generally teach students y=mx+b in middle school and high school. This is considered a perfectly normal part of math education. It's accepted that the students may not understand the full meaning of linearity and regression at first, but at higher levels the formula would be second nature to them. This is the point I am trying to make about statistical learning (not specifically Bayesian theory in particular, but the key concepts around it). Generative ML and stuff like LLMs require a specific frame of mine and view of the world, and it's best to develop it early. The way we are teaching statistical learning theory, at the lower levels, is essentially dumping a massive bag of tricks on them and hoping for the best. We need to change that approach and start treating it like pre-calc and algebra, something to be taught level by level, with building intuition as a core focus. It's not easy to communicate the idea that "the next word prediction of the LLM (depending on the model) is generally the joint probability of all other words that come before it by sampling from a multidimensional probability distribution" to most 14-16 year olds.
I don't see why we should shoehorn these topics into ap-stats. That curriculum is entirely based on classical statistics, which is very useful for what it does.
But suppose there were an ap-ml, where we have word prediction of LLMs, maybe an attempt to teach multi-dim distributions without calculus (ha! I'm seriously skeptical of that, but its your idea), rejection sampling and a suite of easy to code mcmc recipes - MH, Gibbs, Slice Sampling, Importance Sampling, and a dash of Bayesian theory - if there were such a course, it would be quite popular among the high schoolers I work with.
btw, college board does revamp ap curriculum quite often. AP African American Studies was introduced last year. Maybe AP-ML is on the cards.
"AI skills" is comparable to what using a search engine was before.
You'd be absolutely amazed how many people still can't use search engines, other than the absolute bare basics of typing something into google, and giving up if the result isn't on top of page 1.
I've worked with plenty of (non-tech) people that are like fish out of water, when trying to find information. Just learning stuff like boolean opeators, searching for words in quotation marks, specifying which sites and dates to search for, is way beyond what most people know, or do.
Some goes for LLMs. There's a difference between prompts, and knowing what to ask for, and how to structure your questions.
Education is slow, people still don't know how to use a spreadsheet software or scripting language to enhance their lives/work. Students don't use Excel or python to cheat on their homework or exams, so they don't really have reasons to learn those tools.
Meanwhile, user-facing AI tools are often extremely intuitive and feel quite natural to interact with, so by the time young people reach working age they already have familiarity with what they need
In a similar way, the overuse of simplified touchscreen-based devices makes them unprepared for working with computers and other “old tech” permeating the modern office. Many of them do not even know what files are, and their tech skills make them almost completely unhireable.
It's just that for older folks amount of time spent with computing devices was a decent proxy for tech skills. That correlation has probably broken down.
You need to spend time with computers to learn how to use them so if average PC usage drops I'd expect the average skill to drop as well.
They recently got a larger mid-term assignment which involves implementing some well-known, basic data structures, and the standard functionality associated with them.
In one of the problems, they were given skeleton code to a bit more complex functionality - and their task is to explicitly use LLM of their choice to fill in the code, and test if the code works, using a set of tests.
I think the class in general has been updated to assume that students are using LLMs more and more, as the problem sets this year are longer and more complex, compared to those of past years (which were made available to all current students).
I’m reminded that the big thing when I was in college was XML databases. XML databases, we were assured (though not particularly convincingly) were the future. I didn’t opt for the course covering XML databases, and somehow survive 20 years later; meanwhile no-one really remembers what an XML database even was.
(It was, all told, a rather boring time for tech fads. The 90s AI bubble had imploded with such finality that people barely dared utter the term 'AI', the dot-com crash had just happened, and the gloss was off CORBA, so weird XML-y stuff got pressed into service for a few years until the next thing came along.)
AI is improving fast enough that maintaining fluency with its benefits seems like a credible “very good idea” for a lot of people.
I am a software developer and I hire devs as well. If somebody is ignorant of AI or refuses to use it, that is hard pass for me. It is not all that different from an accountant not wanting to use computers. Sure, you could still do some work. But, you will not be competitive.
Or even just redacting their documentation entries.
I’d say the main devs that don’t benefit is mid-level ones. Good enough to do some programming, but not good enough to figure out how to benefit from gpt.
I'd take a collegue with knowledge of the fundamentals and that knows how to ask the right questions - and when not to - over "produces a metric shit ton of code, but when asked doesn't actually know anything".
The first one might be slow and might be editing his, horrors or horros, documentation entries by hand, but that's because he is thinking and refining while he is doing it. He is not an automaton spitting out "work units".
After some contemplation he will wipe the whole project off the table, because in the grand scheme of things it doesn't pan out while collegue number two was busy generation a metric shit-ton of work units and being happy about his productivity.
No LLM will tell you that or more specifically, no LLM will tell you that if you don't know what to ask for in the first place.
Anything javascript related it's probably fine tho.
It's essentially a DoS attack on the maintainers.
Say, you are a python developer and you are working on some FastAPI rest service. Do you 1) ask chat GPT to generate documentation for your API. 2) do this by hand. or 3) routinely skip that sort of thing. 1) would be the correct answer. 2) would be you being inefficient and slow 3) would be lazy. It takes 1 minute to generate perfectly usable documentation. Tweak it a little if you need to and job done.
Bonus points for generating most/all of the endpoints. I did that a few weeks ago. And the tests that prove it works. And the documentation. And the bloody README as well. Slightly tedious and you need to be good at prompting to get what you need. But I managed.
Artisanal software is not going to be a thing for very long. I expect people to get standard stuff like that out of the way efficiently; not to waste days doing things manually.
I would also encourage candidates to use chat gpt during coding tests. If I actually believed in those, which I don't. I'd be more interested in their ability to understand the challenge then to produce a working solution from memory. Use tools for that.
Swagger has solved that one for years now. In any case I think it's foolish to have multiple competing "sources of truth"... in the worst case you have stuff written on your website/Confluence/wiki, some manually written docs on Dockerhub, a README in the repository, an examples folder in the repository, Javadoc-based comments on classes and functions, Swagger docs (semi-)autogenerated from these, inline documentation in the code, years worth of shit on StackOverflow and finally whatever garbage ChatGPT made out of ingesting all of that.
And (especially in the Javascript world) there's so much movement and breakage that all these "sources of truth" are outdated - especially StackOverflow and ChatGPT - which leaves you as the potential user of a library/API often enough with no other choice than to delve deep into the code, made even messier by the intricacies of build systems, bundlers and include/module systems (looking at you Maven/Gradle/Java Modules and webpack/rollup/npm/yarn/AMD/CommonJS/ESM). The worst nightmare is "examples" that clearly haven't been run in years, otherwise the example would have reflected a breaking API change whose commit is five years ago. That's a sure way of sending your users into rage fits.
IMHO, there is only one way: your "examples" should be proper unit/integration/e2e tests (that way you can show your users how you intend for subscribers to use your interface, and you'll notice when your examples are broken because they are literally your tests!), and your documentation (no matter the format, be it HTML, Markdown or Swagger) should all be auto-generated from in-code Javadoc/whatever. Inline code should be reserved for stuff that's only needed to be known for someone intending to work on the actual library, to describe why you took a certain way of implementing a Thing (say, to work around some sort of deficiency) - think of the infamous "total hours wasted here" joke [1].
My to-go example, even if it is not perfect (the website is manually maintained, but the maintainers are doing a fucking good job at that!) is the Symfony PHP framework. With the exception of the security stack (that one is an utter, utter nightmare to keep up with), their documentation is excellent, legible, and easy to understand, and the examples they provide on usage are clear to the point. Even if you haven't worked with it for a year or two, it's so easy to get up to speed again.
[1] https://nickyreinert.medium.com/ne-13-total-hours-wasted-her...
I was referring to both the implementation and generating good/complete documentation strings for that, which is tedious and repetitive work. Obviously, I'm using openapi (aka. swagger); support for that is built into fastapi. That's one of the reasons I picked that.
The tests are also generated but I screened them manually and iterated on them with chat gpt (also test this, what about that edge case, etc.). I know how to write good tests. But generating them is a way better use of my time than writing them manually. Especially spelling out all the permutations of what can go wrong and should be tested and asserting for that. AIs are better at that sort of thing.
These are simple crud endpoint. I've written loads of them over the years. If you are still doing that manually, you are wasting time.
I mean we know how - and why - to write documentation. We have all basic skills and just use LLMs to automate that for us. These skills, however, were won through endless manual labour, not by reading about it. We practiced until we could do it with our eyes closed.
Where will the next generation come from .. ? I appreciate most companies don't have to worry about this, but if I were the head of a very large multi-generational enterprise I would worry about the future of knowledge workers. AGI better pan out or we are all fucked.
Ok. So why generate it at all? Just have the user generate it when needed or automate as part of the build. Seems like option 3, the lazy option, might be the right one.
EDIT.
The point of the comment is not to say that you will in fact have all knowledge available. There's guidelines about how you're supposed to read here on HN.
The point was that this device that people could only dream of a few decades ago actually became available.
Maybe it's worth reminding people that you used to not be able to sit on your butt at home and still access just about any undergraduate level material you could think of, along with a street map of everywhere, as well as a way to take pictures, deal with your finances, and a zillion other applications.