> It was really quite depraved but it worked
> but alas the overworked, underskilled, and somewhat frightened IT person couldn't stomach the risk
this seems like a reasonable call to me.
> but alas the overworked, underskilled, and somewhat frightened IT person couldn't stomach the risk
this seems like a reasonable call to me.
To give some context this was a utility company. The gas pipes were consumer and commercial supply lines. You need to know where they are buried so people don't blow themselves up digging them up. It was safety critical.
>> It was really quite depraved but it worked
The baseline context for this, though, is a very error prone process. It was common for me to register a pipe in the technical specification (size depth etc) system and map that (by rekeying the handwritten ID) and accidentally pick a different location in the system for indicating connection of service to main pipes. You could have a service pipe (to someone's house) registered to a main in a completely different area. My script eliminated these types of errors.
The other thing to note is it didn't change the underlying process. You could, and I did, check the underlying form fills (something I wouldn't bother doing when it took 6 times as long).
I'm not trying to denigrate the individual here, he might have been doing his best given his knowledge and level of autonomy (he might also have been chickenshit) but as an organisation they let some value fall on the floor from an employee because they had no mechanism to capture and integrate it.
He could have written his own program and just used it, and IT would never have gotten involved. Because it was using a 3rd party tool, IT got lazy.
My company, BTW, not only explicitly allows AHK, they actually encourage it. I mean, if you're going to give people compilers to write arbitrary code, this is not worse.
Imho, this IT guy did his job, more or less. What was the testing plan for this AHK script? What edge cases were considered? Unit tests, integ tests.... was there a preprod environment for it? A trial run? Can you safely roll back if / when bad data shows up? What's the retry strategy for network or up/downstream fault tolerance? Is there alarm or monitor coverage for bad scenarios?
I'm not saying a bunch of people typing are error prone, if that's your take on my post. I just think that there's a VERY different set of risks (and therefore mitigations) between a distributed manual process and a large scale automated one.
The IT guy might have on some level recognized this and judged the possible loss to be way more than the "guaranteed loss" of the not-broken current process.
If I were an IT guy with low level skills, I'd be proud of myself for making that hard call.
If a new tool is needed, a steering committee of some kind should be used, and IT will be one part of that.
Work would come to a crawl if IT had to validate every single tool used in our company. The notion that this is within IT's scope is flawed. It's the equivalent of asking a government agency to vet every single product that is produced by any company for consumer safety in the whole of the United States, and then rejecting most of them because of the lack of resources to thoroughly vet them.
If so, then they're completely right to stop some novice programmer from bringing their scripts to bear. There's no reason to take that risk if they have the funding for the current human solution. Bring it up to the superiors and pitch it, talk to some devs. I've seen groups lose hundreds of millions of dollars when one of these excel vbscript python enthusiast AHK things go off the rails. It's a mess, and - again - risk aversion is the most reasonable stance from a business standpoint. He's not a hacker founder, he's a business IT manager.
Edit to say, it sounds like you work at a company with <500 engineers, maybe even <500 employees total. That's fine! It's good even, but it's not an experience that's easily exported to the realm of Actual Business, at least not exactly. As a consultant, I've had billion dollar companies that need the fast fix (python script with a csv datastore to keep some account info or whatever) and I've worked for companies where this sort of approach would get you insta-canned. They're both right enough, but assuming one size fits all is absolutely crazy. It sounds like the mega oil corp above was the latter, at least to me.
As I said in my comments above, it's a question of who owns the data. In general, other than things like financial information and HR records, IT does not own the data in our company. They provide lots of services (version control, Jira boards, etc). They often are responsible for backing them up at a specified interval (daily, etc). And they are responsible for configuring them to be secure (which may involve shutting off APIs). But they don't own whether someone can write a script to automate clicking on fields, etc.
They don't own the data and outside of the agreed backup are not responsible for it. They absolutely would not get in trouble if someone automated something that destroyed that data. The policy is "We provide the service, but you are responsible for what goes into it and how." They do not need to see that we've vetted people's scripts for bugs, etc. They expect us to be responsible. If the data is lost, we will be in trouble, not them.
If IT was responsible for every database, Jira board, etc in the company beyond their regular backup, little work would be done. It's a huge company, and we cannot have one org be a bottleneck.
At times they do need to put some restrictions, but even in those cases, the department is free to get their own hardware and run their own database/JIRA board/whatever. There are a few security requirements to hook it up to the network, but IT will not get in our way if we set up our own Jira board/DB. They'll just tell us that we are responsible for backing things up nor will they provide support if we break things.
This is why my company has lots of pseudo-IT teams across departments. The company wide IT is responsible for the big things. The smaller pseudo-IT teams exist to be more flexible and not be constrained. They don't exist in defiance of IT - IT actually endorses it.
> I've seen groups lose hundreds of millions of dollars when one of these excel vbscript python enthusiast AHK things go off the rails.
But why is IT responsible for it? Perhaps in your company this is in their purview, but not in ours. And I'd argue that it shouldn't be. IT cannot go into every org and police what VBA scripts people use, and if they did they'd likely do more harm than good. While you may have seen losses of hundreds of millions of dollars, I'm pretty sure the gains from Excel VBA scripts is much, much more.
> Edit to say, it sounds like you work at a company with <500 engineers, maybe even <500 employees total.
Nope. Well over 50K employees, with over 60% of them being engineers. Likely the top company in the field we're in.
No one has answered my question that I brought up several times: If I can write my own program to do it, should they ban compilers as well? I'm sorry but whatever damage a VB script can do can also be done by C#. Should we ban Visual Studio?
I don't understand what that means. With noted exceptions, nobody owns specific data in any of the many companies where I've worked or consulted. You own data stores, get restricted access to other data stores... that's services, even if it's SOA/P. Standard for many many years.
> If IT was responsible for every database, Jira board, etc in the company beyond their regular backup, little work would be done. It's a huge company, and we cannot have one org be a bottleneck.
Not sure what this has to do with the situation at hand, where a user who was a part of a human data entry team dealing with metrics around oil rigs and live oil pipeline data decided to automate their data entry dayjob. No data stores or backups were touched beyond their contribution.
> No one has answered my question that I brought up several times: If I can write my own program to do it, should they ban compilers as well? I'm sorry but whatever damage a VB script can do can also be done by C#. Should we ban Visual Studio?
I don't understand your insistence that a language is meaningful. Why would the IT guy sign off on a haskell program, or a java program, or a python or awk or bash pipeline? He doesn't have the expertise to verify that it does what your job is to do. He doesn't know the guy's job, doesn't see testing, can't verify if it's right or not. With no verification of correctness, he's completely right to block it, no? I almost feel like I'm missing something here.
Sure, my employer allows me to run whatever I want on my laptop. They don't let me pump piles of data into prod environments from that laptop even if it's done in AHK, F#, ... whatever. Why would they? That's a crazy requirement to have.
If you have over 50k employees and each person is allowed to script prod data, let me know the name and some prod endpoints for your company. I'll give you a hundred bucks on the spot :)
You keep bringing up the oil fields, oil pipeline, etc. The original commented did not indicate that this is the purpose, and nor did he indicate the importance of said data, or the cost of the loss. I think I have an idea to what he is referring to, and the data is not that critically important. If it has some importance, there will be a paper trail. As another commenter said, they either have a process to audit the validity of data entry or they don't. Whether it was hand entered or automated is irrelevant.
Throughout this thread you've been arguing from extremes, invoking cases where such a thing could be disastrous, etc and using that as a general purpose argument. No one is saying there aren't occasions where IT should not interfere - I've acknowledged that there are cases where they should. There's nothing to indicate that the original commenters use case is one of those.
And this is what people in the thread are caring about: Someone who has no idea about the problem interfering with a perfectly fine methodology.
I could go on and address your other points in this and other comments, but I don't want to discuss this with someone who doesn't seem to understand the problem domain at all.
There's not been a response to my question of how you reconcile things after this script messes up for the first time; how does that happen? What's the DR plan? if you have real business and real clients working on this data, tell me what the plan is; it shouldn't be that hard.
Once again, if I wrote my own Python script they would not be involved, and my Python script is as error prone as using AHK.
Think of AHK as similar to a compiler + linker. By rejecting AHK because someone can write a program that could screw things up, you might as well just prevent people from writing scripts and compiling programs.
> I'm not saying a bunch of people typing are error prone, if that's your take on my post. I just think that there's a VERY different set of risks (and therefore mitigations) between a distributed manual process and a large scale automated one.
You've essentially argued that IT should oppose any automation that is not thoroughly tested. And how will they determine how well it is tested? Do they have those skills?
Where I work, IT is more concerned about security (viruses, theft, etc), licensing, and things that can break the machine. They don't prevent people from using APIs to mass edit databases as long as you do it to data you own or have permission from the owner. They don't own the data, so you're free to screw it up if you're careless. Ensuring my program is bug free and well tested is the job of my team/org/data owners, not IT.
You're merely pointing out that this IT person has been given more power than his skills warrant.
> You've essentially argued that IT should oppose any automation that is not thoroughly tested. And how will they determine how well it is tested? Do they have those skills?
In large scale, business critical applications? Yes. All everything should be thoroughly tested and/or canaried, with several levels of validation before production releases.
> You're merely pointing out that this IT person has been given more power than his skills warrant.
I think I'm pointing out that his job is to be a safeguard against rogue wanna-be programmers whose ability to destroy things far outstrips their ability to test for things (or even think about edge cases). If that's true, they've done a fantastic job.
A lot of programmers, or programming enthusiasts, thumb their nose at IT staff who naturally push back on stuff like this. In their mind, to play devil's advocate, they're seeing IT as unnecessarily burdensome, and getting in the way of automation.
The IT staff are probably naturally welcoming to the general idea of automation. However, to elevate something up from an off the cuff script that's described as "depraved" to the level of an actual business tool that can be relied upon, is a HUGE time and money sink. Validation is a very complex problem as you have described. This is the difference between consumer and enterprise hardware. If business management isn't setting this as a priority for IT, and if IT is already overwhelmed with the "known" quantities of work, then it just isn't practical to officially adopt.
I see it like this: It takes 1 "effort point" for a data entry staffer to manually enter the information tediously. It takes 10 effort points for the same staffer to write the AHK script to automate it. (Good for the staffer, assuming there's no flaws, and their reward is a longer lunch break.) To truly validate an automation as a business tool would take something on the scale of 150 to 200 effort points and involve lots of conversation - assuming there even is a framework for new tool validation established that can be used. IT knew this and made the right call. If company management wants to adopt a new tool without research and validation, that can be reckless.
Which raises the question, why does such a person even have the authority to make calls on what software can or can not be used within the company? You shouldn't have the right to make these kind of decisions without basic understanding of how software works (beyond "I click stuff and things happen, magic!").
> What was the testing plan for this AHK script? What edge cases were considered? Unit tests, integ tests.... was there a preprod environment for it? A trial run? Can you safely roll back if / when bad data shows up? What's the retry strategy for network or up/downstream fault tolerance? Is there alarm or monitor coverage for bad scenarios?
Without understanding of how software is built, those are questions the IT guy can't possibly ask thoughtfully. They become a mere checkbox ticking exercise. And most of them don't apply anyway.
> If I were an IT guy with low level skills, I'd be proud of myself for making that hard call.
It's not a hard call. It's the easy call. The ass-covering, responsibility-mitigating, work-minimizing, consequence-free default deny call.
> Unit tests, integ tests.... was there a preprod environment for it? A trial run? Can you safely roll back if / when bad data shows up?
Nothing of the sort :)
But... Let the adequate be the enemy of the less awful?