How We Engineered CMS Airship to Be Simply Secure
paragonie.com
paragonie.com
Take a look at the list of security concerns Airship tries to manage. Most can be avoided by separating the content creation aspect and the publishing platform. Content creators represent the smallest audience group your site has, why then have a full application exposed to the internet running only for them to easily change content? Content creators should have an offline app, make changes, press publish and have those changes uploaded to a static server. The whole CMS control panel paradigm seems kinda flawed to me.
An offline app would cause a number of its own problems in that regard, because now you're actually having to design a backend that interface with your webserver and puts the content to it. What happens when you have multiple editors? You're now pulling information from a central resource, which is going to be over the internet, whether its on a different server or not.
Not to mention, by having it 'offline', you can't have mobile apps to edit content on your site.
Static definetely has advantages, but when scaled and working with data its not that simple.
- Offline editing for online publishing
- Static mirrors to resist censorship in the face of DDoS
Aside from the API that receives data and triggers an update (which is intended to be set up on a different vhost), the attack surface will be minimal.
The only reason this isn't already a thing is that I've had other projects that were higher priority, including one I'll be releasing in a month or so (but probably not submitting to HN; it's an entirely commercial project).
In fact, while I found the article interesting I could have done with a bit less of the barbs about other frameworks. If the only way you have of building up your own tech thing is by tearing down other people's ...
This one made me think of engineers tinkering with mechanics on a giant flying airship :)
> chown -R myusername:www-data airship
> chmod -R g+w airship
Have you ever wondered why some people say that PHP is insecure? That's one of the major reasons.
This app claims to care about security, but at the same time it self-updates and has no external code integrity control. This means that if the webapp has a vulnerability which was exploited, it will be very hard to detect it -- was this file changed by auto-updater or by malware? you don't know. And even if the new version of webapp was released which fixes the bug, you can not be sure that infection is gone -- because the update process may also be compromised. And you cannot easily remove code dir and re-download -- the settings are mixed-in with the code.
Compare it with any other stack -- Ruby on Rails, Django, JAR files, whatever. Code cannot modify itself. The only writable thing is the database. Additionally, the repository is often under git control. If there is a vulnerability, update software and restart the server (if in doubt, backup the database and nuke/rebuild the server instead). Even if you made a configuration error and left code dir writable, "git status" is enough to tell you any changed/added files.
And this is why you, as an admin, should avoid PHP if you care about security. Sure, you can use sane techniques even with PHP code -- I am sure that's what Facebook and Wikipedia do -- but most random PHP projects will require writable code dir. When choosing a web app, choose non-PHP one first.
It also convers the "external code integrity control":
Keyggdrasil makes sure everyone sees the same public keys, prevents anyone from rewriting history, and makes targeted attacks noisy.
Autoupdating is not a feature specific to PHP. You can do manual updates as you described with it too, so I'm not sure why you even contrast it to other languages or CMS.
It's pretty much established that properly implemented autoupdates increase security, not reduce it. I agree that writeable and executable directory is a huge risk, though.
https://paragonie.com/blog/2016/10/guide-automatic-security-...
Security at the expense of usability, comes at the expense of security.
CMS Airship went with the "self-writing code" option because we believe the potential damage of a 1day vulnerability to be much a higher concern than theoretical "this isn't really following best practices grumble grumble".
By all means, don't chmod/chown anything, and run continuum.sh as a more privileged user if that's what you want.
Just don't disable automatic updates.
Because there will be security vulnerability, and people will be exploited, and even if your update system runs every 4 hours, it will take dev team some time to write up a fix and push an update. And obviously, the first thing attacker would do will be to disable any integrity checks your auto-update does. So in the most common case, the host will stay compromised forever.
This looks pretty horrible to me. I am appealed that you call this "theoretical". But then, (in my experience) the security is not a strong side of PHP community.
The sad part, it is not hard to fix. You already have separate update script -- obviously "don't chmod/chown anything" is not enough, you at least want the cache directory to be writeable, and while I have not looked at the code, I suspect there may be other locations that web app will want to write to. You can change your Dockerfile to use secure setup. It will take a bit of work but it is definitely possible.
Until this is done, "simply secure" is only until the first compromise.
The auto-updater does nothing to make this worse. That's true of any vulnerable system. Once you're owned, you're owned.
> Because there will be security vulnerability, and people will be exploited, and even if your update system runs every 4 hours, it will take dev team some time to write up a fix and push an update.
It runs every 15 minutes, in the default configuration, and our usual turnaround time for getting a bugfix out is less than 1 hour.
> The sad part, it is not hard to fix.
It's not broken, it's a trade-off. Nothing to fix.
> You already have separate update script
That's the kicker: If your threat model prioritizes "don't let the httpd write to the web directory" higher than your usual non-power-user, then simply:
1. Set up a cronjob to run the update script, run by a more privileged user.
2. Don't chown/chmod.
The reason we do this by default is because our threat model is ensuring automatic updates work for everyone (including people who are not sophisticated enough to set up a cronjob).
> You can change your Dockerfile to use secure setup. It will take a bit of work but it is definitely possible.
I guess?
The Dockerfile was provided by the community and is maintained by the community; I don't use it nor do I touch it.
I'm not a Docker user. https://news.ycombinator.com/item?id=10269200
> Until this is done, "simply secure" is only until the first compromise.
That's literally true of anything. If you get compromised, burn the machine and recover from a backup.
No. If you have RCE in the regular app, the attacker cannot make attack 'stick'. Once you reboot the machine, or restart service (in case of systemd), any malicious code no longer runs.
Now, if there is a privilege escalation bug the situation will get worse, but having 2 unpatched bugs is significantly less likely. Of course, you want to make sure unattended updates are enabled so your host OS is up to date.
As a result, most other systems will actually self-heal from minor compromises. Only your system (any many other PHP systems in general) will stay compromised forever.
> It runs every 15 minutes, in the default configuration, and our usual turnaround time for getting a bugfix out is less than 1 hour.
1 hour from what? From someone sending email to your team? If a black hat finds an exploit in your service, they will start compromizing the existing web servers first. You will not find out about the bug until someone notices something weird on their installation, and that will take way more than an hour.
> 2. Don't chown/chmod.
Have you actually tried this? Looking at your github, there seems to be `src/tmp/cache` directory -- presumably at least this one should be chmod'ed? What about others? What about plugins -- would they work fine without writeable root?
I have tried to make other PHP packages use immutable code, and it was very flaky and painful. You need to structure your program in a certain way to make sure it works with immutable root -- one location for temp, another for configs, third for the code. If your main development team uses writable code, there is a high chance immutable code would not work well.
> It's not broken, it's a trade-off. Nothing to fix.
> The reason we do this by default is because our threat model is ensuring automatic updates work for everyone (including people who are not sophisticated enough to set up a cronjob).
Why do you say it's hard to set up a cronjob? It is simple -- your deb/rpm package just includes file /etc/cron.d/aristrip, and voila -- you are done. The only place where cronjob setup is hard is on these cheap shared PHP hosting services. And I think no one should be using them -- get a $5 Linode VM instead.
The thing is, the modern Linux systems went pretty far in terms of security and isolation. You have multiple easy-to-use technologies for threat protection -- docker, Apparmor, capabilities, systemd' *Path, etc.. They work pretty well to make sure that attacker which got RCE cannot take a hold on a system and elevate privileges.
Except your system is not compatible with any of them, by design. Because you don't care. Maybe you should change your motto from "A Secure Content Management System for the Modern Web" to "A Secure Content Management System for the Cheap Shared PHP Hosting", because you can certainly do much better for the modern web.
As ever with PHP sites validation seems to be a mess. For example we've got a `escapeSupplierName` function[2] used in some places[3], but a different regex used when creating a suppier[4]. Variants of that regex also appear all over the code, why not put it in a single place?
Properly escaping SQL injection is great (and something PHP apps seem to have a continual problem with), but why did you decide to roll your own framework for doing this? Are all of the well maintained, tested and great ones already available not suitable for some reason? It's just your app turns into a hard to check, inefficient[5] sludge, full of lots of code for handling framework related things like an ORM, caching[6] (that code is practically duplicated in a lot of places[7][8][9][10]), escaping[11], mime whitelisting[12] etc and not much about a CMS. This is where security bugs appear.
1. https://github.com/paragonie/airship/blob/eb4293aee5be59e329...
2. https://github.com/paragonie/airship/blob/c4c9384d4d7860738d...
3. https://github.com/paragonie/airship/blob/c4c9384d4d7860738d...
4. https://github.com/paragonie/airship/blob/95af23ca782e8ecb10...
5. https://github.com/paragonie/airship/blob/63cf0661ba21cbb3f3...
6. https://github.com/paragonie/airship/blob/master/src/view_fu...
7. https://github.com/paragonie/airship/blob/master/src/view_fu...
8. https://github.com/paragonie/airship/blob/master/src/view_fu...
9. https://github.com/paragonie/airship/blob/63cf0661ba21cbb3f3...
10. https://github.com/paragonie/airship/blob/2a8a87934921ecda85...
11. https://github.com/paragonie/airship/blob/eb4638eb31510028f4...
12. https://github.com/paragonie/airship/blob/eb4638eb31510028f4...
1. Context matters: That's a CLI script used to build and sign core updates. We bundle it with the source code so anyone can fork our project and use only their own keys, easily.
2-4. This is a good point. https://github.com/paragonie/airship/issues/181
> Properly escaping SQL injection is great (and something PHP apps seem to have a continual problem with), but why did you decide to roll your own framework for doing this?
Because when Airship was started, everyone insisted on backwards compatibility for PHP 5 and I wanted to make use of strict typing. The master branch includes a static analyzer as part of continuous integration.
> Are all of the well maintained, tested and great ones already available not suitable for some reason?
Have you seen what I've done to PHP frameworks over the years?
https://scott.arciszewski.me/open-source/
https://paragonie.com/security/advisories
I wasn't in a hurry to pick up other peoples' technical debt, especially if we're paying money for vulnerabilities: https://hackerone.com/paragonie
5. That's simply a paranoia/convenience feature. If you've got a better way to automagically convert filename.txt into filename-5.txt if filename.txt, filename-2.txt, filename-3.txt, and filename-4.txt already exist, I'm all ears.
6-10. Good eye. I think that can be refactored to repeat itself less.
11. I don't see a problem with this?
12. I don't see a problem with this either.
Ahh, my bad, it's still an odd way of doing this. And it's vulnerable to injection via the prompt parameter (not that it matters, but...). That pattern is repeated in a couple of places like the updater.
> Because when Airship was started, everyone insisted on backwards compatibility for PHP 5 and I wanted to make use of strict typing. The master branch includes a static analyzer as part of continuous integration.
Strict typing is great, but you don't have to write your own framework to benefit from it. Is PHP 5 not compatible with modern frameworks?
> Have you seen what I've done to PHP frameworks over the years?
> https://scott.arciszewski.me/open-source/
> https://paragonie.com/security/advisories
> I wasn't in a hurry to pick up other peoples' technical debt.
Indeed there are some vulnerabilities found in PHP libraries in those pages, but hardly enough to warrant your position.
You aren't in a hurry to pick up others technical debt, but you rush ahead and create your own?
All I'm saying is you put out a post saying you designed this from the ground up to be secure, but did you really? Is rolling your own ORM and homegrown framework secure? Says who?
Yeah, you use modernish stuff like csp etc, but the code is still a typical PHP spaghetti of mixed concerns and hard-to-audit flows. It's not really a CMS, it's a NIH syndromed framework with a CMS on top of that. That's a typical source of bugs.
> That's simply a paranoia/convenience feature. If you've got a better way to automagically convert filename.txt into filename-5.txt if filename.txt, filename-2.txt, filename-3.txt, and filename-4.txt already exist, I'm all ears.
It could be done in a single query, I'm on my phone so i cant write you an example right now.
> 11. I don't see a problem with this?
> 12. I don't see a problem with this either.
They are both framework functions, and you wouldn't be writing them if you had written this from the ground up to be secure IMO. It's kind of code smell. Using a whitelist is good though.
Those are all my research findings. :P
> Is rolling your own ORM and homegrown framework secure? Says who?
Says the person who routinely finds exploitable vulnerabilities in other PHP frameworks and content management systems.
There will always be things to improve. Feel free to try to find an exploitable security hole.
> It's not really a CMS, it's a NIH syndromed framework with a CMS on top of that. That's a typical source of bugs.
It's not "NIH syndrome" it's "I don't trust any of these people to write a secure framework and I have the experience to justify this concern".
> They are both framework functions, and you wouldn't be writing them if you had written this from the ground up to be secure IMO.
Escaping-on-output still implies that escaping happens.
Yes, and well done! But there are not many framework specific issues you've found. None in Laravel for example. Is your homemade framework superior to Laravel? If so, how?
> Feel free to try to find an exploitable security hole.
I can't be bothered to learn your home grown half framework so i understand the convoluted logic. That's the point. There are vulnerabilities in there, I've seen enough code like that to almost guarantee you. But like a lot of PHP projects that roll their own everything it's no trivial task to do a code review. If you had used a well made framework then you would have got rid of a lot of code that only you really understand.
> "I don't trust any of these people to write a secure framework and I have the experience to justify this concern".
Your code doesn't look like well written, secure code. I'm sorry to sound abrasive but it's true. You mix validation in random places, perhaps inconsistently, and your project tries to do everything. This is not how secure projects are typically designed.
> Escaping-on-output still implies that escaping happens.
It's only used in 8 places as far as i can see, in some seemingly random places.
Our cryptography is certainly superior to Laravel's.
https://github.com/illuminate/encryption
https://github.com/paragonie/halite
Beyond that, nobody has paid me to look deeper and I haven't had any reason to.
> There are vulnerabilities in there, I've seen enough code like that to almost guarantee you.
If so, then anyone reading this thread who's aware of any vulnerabilities will be interested in: https://hackerone.com/paragonie
Ok, but cryptography is only a small portion of what a framework does and not necessarily the most important bit. I was asking more about your routing, view layer, ORM, code layout, templating, validation, caching, ACLs etc.
> Beyond that, nobody has paid me to look deeper and I haven't had any reason to.
Look, you can't put up a blog post titled "How We Engineered CMS Airship to Be Simply Secure" and say that when asked "well, how did you actually engineer it to be secure?". In your post you talk about problems all solved by a framework, and apparently you haven't even evaluated what is out there already and rushed headfirst into building it all yourself.
This is not engineering a project to be "simply secure", in the sense that it is definitely not simple and probably not secure!
> If so, then anyone reading this thread who's aware of any vulnerabilities will be interested...
This is great! I wish more companies did this. However, as stated previously, the ad-hoc nature of the code and the single-developer nature[1] makes this pretty hard to do from a code review point of view. Again, using a framework makes your code way easier to review and a lot smaller to boot. Anyone with Django experience can pick up a Django app and start to review it, rather than have to learn whatever bespoke framework the developers of the project dreamed up.
Also I just wanted to add that I don't mean to come across as a dick, your project looks interesting for sure and a lot of work has gone into it. PHP CMS's need to start taking security seriously, things like Joomla and random Wordpress plugins are a blight and anything taking security seriously is great! As someone who works in the security industry it's just a bit tiring to see yet another homegrown PHP half-framework full of it's own intricacies and potential issues with the ever familiar problems described in all my comments above, coupled with a post saying how it was engineered to be secure from the ground up.
1. It's hard to tell, in this graph (https://github.com/paragonie/airship/graphs/contributors) is 'paragonie-security' a single person?
I've looked at Laravel, about two years ago. http://seclists.org/fulldisclosure/2015/Apr/57
I haven't looked deeper at what it's done since then, since nobody is paying me to do so. I wasn't impressed then, and instead of waiting for them to get around to doing things right (while figuring out how to preserve backward compatibility), I started from zero.
You don't have to like that decision, but that's the decision I made.
> Again, using a framework makes your code way easier to review and a lot smaller to boot.
Sure, use a framework.
And then this happens: https://kivikakk.ee/cryptography/2016/02/20/breaking-homegro...
I designed my own because, unfortunately, most of the PHP community either doesn't prioritize security enough to be trustworthy, or is simply in over their heads.
We use third party libraries (Twig for templates, HTML Purifier for allowing some HTML but avoiding XSS vulnerabilities) because they're well-studied and trustworthy libraries.
However, I do not trust most other PHP developers to do anything security-critical correctly. And thus, I will not design something meant to be secure based on an insecure foundation.
If your only reason for Airship's status as "probably insecure" is because it's "ad-hoc" and/or "bespoke": Sorry, that's insufficient. Got a PoC handy?
Otherwise: we're talking about pure hypotheticals that are far removed from the real world and there's not much point in that. You can frame your criticism as "things to make the code easier to read/understand" and whatnot, and they stand alone. Citing them as "probably a vulnerability" is questionable, however.
> It's hard to tell, in this graph (https://github.com/paragonie/airship/graphs/contributors) is 'paragonie-security' a single person?
For the moment, I'm the only technical person at Paragon. That will change, and that account will be delegated to signing/committing the code changes made by other employees.
> As someone who works in the security industry it's just a bit tiring to see yet another homegrown PHP half-framework full of it's own intricacies and potential issues with the ever familiar problems described in all my comments above, coupled with a post saying how it was engineered to be secure from the ground up.
You've raised legitimate concerns over:
- Apparent NIH-syndrome, which in this case is justified
- Minor code quality nits, which are now fixed
- Bus factor