I can recommend Snyk[0] (I am not affiliated, have used their products with clients before). They've got a fully functional free tier, and they have a ton of relevant features. And yes, there are other, competing products out there you could use.
529 karma · joined January 2, 2018
You've mentioned many issues you've had with your Python code base. Would you kindly expand on how/why C++ or Rust solve those in your particular case?
You mention this as a case against using Python, but this is entirely unrelated to Python as a framework. In other words, had they used PHP or Node or %s, they'd still have to face the issue of optimizing the request-to-response path.
This relates much more to the issues that often come with the common practice of separating your webserver, from your business logic (Flask, in our case).
> "so you have to use some asynchronous tooling"
Solutions such as mod_cgi and wsgi have been the standard for years at this point. That's, in many ways, a solved problem.
> "In order to get off the ground, you also need to use pip, which causes an inordinate number of problems vs. the bsd ports tree or .deb files."
Any specific issues you can expand on? Because this is a rather vague form of criticism.
Using pip is ridiculously easy. I've been using Python extensively in various production environments and rarely had to point a finger at pip. Package and dependency management in Python are solid (that isn't to say things can't improve).
> "You can use virtual environments for each script, but then patching zero days becomes infeasible."
Again, this is an entirely subjective form of criticism. I haven't had any issues with keeping my virtualenvs up to date.
> "If you want basic type safety, you need to rely on external tooling."
Again, you just seem to favor strongly typed languages and that's fine, but that's hardly a point against using Python here. Type safety is simply not a Python thing, much like weak typing is not a C thing.
> "In my experience, third party python libraries evolve types and api’s without warning. This shows up at runtime, so it introduces security holes."
Developers will sometimes break APIs. How is this a Python-specific issue? These things are literally happening everywhere.
Radare, in my opinion, is mostly lacking on the user-experience sides. IDA is easier to pick-up and use. Also, I feel Radare's decompilers don't fare as-well as Hex-Rays. I equate this to Windbg - your best Windows debugger, which unfortunately is relatively tough to pick up (I'd argue, mostly due to poor UI choices). Forgoing that, Radare is absolutely packing.
I've moved on to reverse most of my projects with Radare. I'm still missing a handful of small features/plugins from IDA, but it's not a big issue.
> "* Reference counted allocator (basically, get libtalloc and roll it in [1])"
Why and how should that be standardised exactly? Memory allocation is platform-dependent, hardware-dependent and generally case-specific. malloc() and free() are the lowest common denominators the standard can assume, anything beyond that is simply restrictive. If you need a "reference counted allocator", why not just find/implement one that simply suits your needs?
> "* Coroutine standardisation would be awesome (stack swap primitives, with boundary checks etc)...."
Again, what makes you think this can be standardised across the infinite span of platforms and compilation-targets, where C is often used?
> "* In fact I want SIMD. I don't need these complex types."
I'm not following your point. You're simply asking for a better abstraction for SIMD. Also, as I'm sure you're well aware, SIMD is not available everywhere. Wherever available, you have clear instruction-set APIs/ABIs you need to follow to make it work. What else is missing?
What's stopping anyone from randomly filing GDPR claims against every other website? How many such claims can the system handle at any given time before succumbing to years-long queues?
You'd still be able to detect new addresses and block as necessary. I don't see your point, of course the solution proposed isn't entirely hermetic, but how does your suggestion overcome this exact issue?
> "This kind of approach only works if you are happy to pay the bandwidth price"
Precisely the reason why most people would rather opt-out of serving malicious requests.
(1) Being able to identify the IPs from which attacks originate, in real time. You will need to blacklist those hosts (for a limited period of time).
(2) Minimizing and limiting the impact of traffic you previously flagged, on your infrastructure. Ideally, as hostile traffic is flagged, a response is never sent (at the TCP/UDP level). You absolutely do not want to serve any flagged requests. Block traffic as early as possible.
Well, people are still free to maintain and improve those older tools and bring them up to par with newer standards, if they want to.
Others would argue it's time to start phasing out code and so-called "standards" we've had since the early 90s in spirit of refactoring various elements in the ecosystem. As with most such efforts - lots of code will be thrown away, some of it will be re-used and re-purposed. I'd argue that in the long run - it's probably a necessary intervention.
It doesn't automatically mean "good" either.
> "in its pursuit to be Windows"
Would you please elaborate?
Switching to Erlang won't inherently make those issues disappear. Erlang could definitely save you some C/C++ boilerplate code, but if your business logic is still stateful, it's not necessarily going to be easier to implement any of that with Erlang.
Erlang is probably more forgiving on the developers than C/C++ are, but that can be said of many other languages (ie. Python, Java).
If you're looking to pass messages around between processes/threads, there are plenty of libraries you can use today to start refactoring that C/C++ code:
https://www.rabbitmq.com/ (most likely an overkill for the specific use-case you described, will be more relevant in a distributed computing scenario)
No, that's not my argument at-all. That's just your personal interpretation of my words.
> "You’re also claiming people are rightfully concerned."
I'm not "also claiming". That was the sole claim from the very start.
> "Where is that right coming from? From past experience? Or is they just baseless concerns?"
It's literally in the comment:
(1) Some elements of the GDPR are up for interpretation.
(2) There's currently no case law surrounding GDPR.
If you take both of these facts into account - it is perfectly plausible for people to be concerned, as there's no telling how things will play out in a court of law.
I disagree with the author's lenient and dismissive take on people's genuine concerns. Interpret it as you will.
> "This post is an attempt to calm the nerves of those that feel that the(ir) world is about to come to an end"
This post is actually a single person's viewpoint, a mere speculation of how things may or may not turn out to be. Your mileage may vary.
Wrong. Phones can be triangulated by the carriers regardless.
(2) From a job seeker's perspective, I don't want to be secretly spied on by a potential employer. I can (and often will) volunteer any information necessary, through a mutually agreed upon channel. I wouldn't want someone prying through my personal social network accounts, secretly contacting random people that may or may not know me from past workplaces. I wouldn't want a potential employer making baseless assumptions based on my social media accounts (or lack thereof).
This is just random FUD.
(1) AMD boards are vulnerable to Spectre - https://www.amd.com/en/corporate/speculative-execution-previ....
(2) There are other vulnerabilities that affect AMD boards - https://community.amd.com/community/amd-corporate/blog/2018/...
(3) Vulnerabilities are found everywhere. That's not the first time vulnerabilities are found, and it won't be the last. How is that an argument for anything exactly?
I don't think you are wrong. Timing APIs are tricky and complicated. The small print really matters. Experienced engineers will know this, and so the answer in this instance, as annoying as it may be - absolutely RTFM. Also, funky documentation/APIs exist everywhere, unfortunately.
Quoting the article:
> "You're a new coder, and you find yourself tasked with taking a Unix time_t type value and turning it into some kind of human-readable format"
I'd argue that's exactly the kind of mistakes inexperienced coders are prone to make. Hence we have code-reviews.
In my opinion, as time goes by and advancements are made, the coupling should grow weaker - and so we'll reach a point where there would be a more clear distinction between the two tracks, and they won't share much of the same curriculum, similarly to where we stand today with CS and electrical engineering.
One of the key points I'd like to raise is the (unsurprisingly) substantial performance degradation caused by tracing every syscall/ioctl a process makes. I submit tracing tens (or hundreds) of processes with ptrace/gVisor simply won't fly. Tracing the syscalls alone is expensive, let alone applying any other intricate mid-hook logic.
http://www.linux-kongress.org/2009/slides/system_call_tracin...
> "gVisor is a userspace application and is not itself tied to kernel implementations the way a kernel module would be"
I was referring to the never-ending chase that comes with having to keep tabs on any new/existent ioctls/syscalls. Ioctls are device/driver/hardware specific, which complicates things further.
> "This is true of existing container technologies. An application running under Ubuntu on bare hardware will potentially not run in an Ubuntu Docker image. You'll need to test it extensively."
There's a similarity, but solutions like VMWare/VirtualBox/hypervisors are fighting to be as transparent as possible to the underlying software. That makes things easier on software developers - as we don't all have to spend our time testing those products.
It would appear that gVisor is fundamentally different. It intercepts and tampers with the various syscalls a process makes with the sole purpose of affecting the underlying application - ie. failing a syscall that would otherwise succeed.
Imagine some applications expect the syscall to actually fail (ie. some odd way to test permissions).
I fail to grasp how this is a strength. Pulling the rug from under applications is dangerous. You're in a direct battle with internal implementation specifics - you don't want to get into that as an abstraction layer. You don't want to tailor various hacks for specific applications.
That's not how access control works (on most systems). The system calls are still issued from a single process. If the gVisor process is running in the context of a non-privileged user, the system calls will fail regardless of the codepath.
TLDR; This is a user-space process that hooks syscalls/ioctls made by your "containerised" applications.
(1) This is hardly a strong security model. Proper security cannot be guaranteed by simply hooking API calls in user-space alone.
(2) With this framework in mind, developers now need to worry about yet another layer of indirection. Assume <application> was tested to work on Ubuntu, that fact alone is not sufficient to assume it will keep running under gVisor.
(3) I would personally like to see more documentation/benchmarks regarding the performance impacts that come with using this.
(4) This is strongly coupled with internal Kernel implementations. It will not be easy to port and maintain this across different Kernels.
> "but much like a VMM, it will not allow the application to directly control the system calls it makes."
From user-space? hold my beer.
I disagree. I submit the whole reason we pass objects by reference or by value in the first place, has much more to do with conserving computational resources/reducing runtime, than with providing abstract programming semantics for software developers.