Instead of deleting account, NYT appends ‘1000’ to username and email address
twitter.com
twitter.com
They were still logged in and refreshed the page; they found out by going to their user settings.
Changing object name data though is a terrible practice to implement this.
Otherwise imagine how easy it would be to violate the deletion request if you're running a business and can't remember the names of the people you had deletion requests for. Their data could come up again through normal channels and you'd treat them no differently than another sales contact, thus violating GDPR.
You catch a lot, with catchall: dataleaks, hacks, sneaky data sales etc. When suddenly you recieve, say, marketing mail for shirts on 'jeansonline@example.com' something fishy is going down.
Having done enough security audits, "this was legacy" is a BS excuse. I will go ahead and assume that NYT have an audit department. And that audit dept runs throug the full audit universe every 4-5 years. Someone would have captured that a long time ago (1st, 2nd, 3rd lines)(external auditors)(any sales pitch: "we have 438264728 subscribers")
I call BS. They got busted and now they most likely change this from 1000 to 2000 and call it a day..
Ps: bemis is not my real last name.. but I am using a super cool name over here!!
And in a naive relational database implementation, deleting a user would cascade and delete activity associated with that user.
The easiest way around this is to do soft deletes where the data stays in the db, but the flag deactivates the user's account. Looks like the NYT just did a poor implementation of a soft-delete.
On the other hand, "is deleted" flags end up causing issues when you forget to put "where not is_deleted" in your queries.
Lately I've faced kind of an inverse situation - I have a system that I can't control where things are permanently deleted once in a while for multiple reasons (rogue users, aging out of old versions) and so as I accumulate information in a little data warehouse for reporting, I decided to implement an "is deleted" flag there. Eventually though, deleting from the source was turned off because it's really not necessary.
P.S.: Random advice to anybody working on enterprisey stuff:
1. "Deletion" is too vague and broad. I strongly suggest you call it "deactivation" and some other word like "purging."
2. Deactivation is typically what a company actually wants, even if they don't know to ask for it. By phrasing it that way you also encourage stakeholders to think about "reactivation" before it becomes an architectural problem.
3. True purging is rare, and tends to be related to either disk-space issues or legal requirements. In the latter case, you'll want an audit-trail or tombstone of some sort, meaning it's still a real workflow and not just an easy SQL DELETE statement or something.
It's not perfect, but I find using a library (either directly for inspiration) is often a shortcut to learning about a lot of edge cases from doing your own things.
My solution would be a view for every table. Are there drawbacks? Other solutions?
Where I am now, (as far as Oracle goes) you can't even create your own tables under your own schema. A view requires a meeting with a DBA and their manager and really special, compelling arguments.
I don't understand what you mean re "access". RLS is what removes access to those rows, that's the point of RLS.
In postgresql, anyways, superusers aren't subject to RLS and the table owner, by default, isn't either. But RLS can be enforced for the table owner by a single alter statement.
That's just how badly the system is designed. Write a script to delete all relevant data and the user and it would be a simple op.
> forget to put "where not is_deleted"
Use views.
On the other side, a customer got really pissed off by an online shop we maintained for a client, and asked for his data to be annihilated, we thought "What a douche.".
When you delete a row, every inbound foreign key constraint has to be checked to look for any rows that refer to the deleted row, and most likely you didn’t set up an index for the foreign key, so now you have a table scan. Possibly several.
It’s not that much more work these days to set up a partial index on the table instead and add another WHERE clause, you have smaller problems with people accidentally deleting the wrong thing, and you’ve started down the path to audit trails.
Once you have hard delete, this solution is almost trivial and by far the most user-valuable.
This seems to violate GPDR, no? Attacker attempts to create an account (say: victim@gmail.com) on AshleyMadison and is prevented because the server tracked past users. Attacker could them demonstrate victim@gmail.com was at one point a user on AshleyMadison.com
This is exactly why most password reset emails say "if you didn't request this, please let us know, as someone may be attempting to access your account".
This is also one of those situations where people often put too much shit in the user table. "we have to delete the user row" -- I mean, you have to delete some of the user row, yes.
I like to solve this by proper namespacing. Suppose you instead deliberately have an authUser table which just has what you need for auth -- a UUID to hook into the rest of the system, salts and passwords for direct logins, maybe a nullable date "banned_until" if you want banning; assuming you use crypto bearer tokens rather than an auth tokens table then you also want a column with a date date for "tokens last reset on"; etc. You can put the username in there just fine, that's needed for auth. Maybe you let people log in with email+password and thus you also put their email address in there, also fine.
As long as the authUser table does not grow to encompass all of your other business logic you are good. Other tables foreign key to authUser and you delete rows from them and that doesn't upset the foreign key. You leave the row in authUser to indicate that the username is taken.
An additional "deleted" field on authUser can be used to block logins and thus the username is taken but they can't log in. As for the email address, even if you insist on a UNIQUE and NOT NULL constraint for it (and I would find this surprising in an age where we log in a lot with social media) you can auto purge by setting it to CONCAT(id, "@purged.example") and then you have a valid email address which is nowhere else used in your auth flow, no personally-identifiable information at all. Heck then you don't even need the boolean flag if you would rather forbid the .example TLD from logging in.
So that has worked well for me in the past and it seems to solve those sorts of problems with only a little tweak. The key is that the PII need is to delete the "user row" but that does not have to be the authUser row -- if you separate the two rows out then you can leave the authUser row while still having a table appUser which lives in your application and contains all the cool stuff about this user using that app. It also naturally lends itself to you thinking about a sort of SSO for all of your different applications up-front.
You're obligated by GDPR to disclose to affected parties that their data has been compromised, but you were also obligated to delete the data by GDPR.
It can get complex (for example foreign keys need special handling).
One option is table partitioning: https://www.2ndquadrant.com/en/blog/postgresql-12-foreign-ke...
You partition the table based on deleted or not, and then query either both tables together, or just the active table.
Read article 17:
https://gdpr-info.eu/art-17-gdpr/
And previous HN discussions:
Even what you can delete can (and will) survive in logs and backups, web archives, screenshots, etc. Deleting things on the Internet is just difficult.
Is this even legal? I've never heard of a company letting an outside firm go through their database to confirm any sort of statistic like that. Who is doing this auditing?
God forbid the needs of the user harm the interest of the investor.
Some violators might successfully keep their assets out of the reach of EU enforcement, but that's going to be really tough to do for any large business with global operations.
How that messes up your technical implementation is your problem
Well that's an understatement. NYT has the tech resources to do this a million better ways.
> investor quarterly report [...] active users
Speaking of "active user" counts, it's convenient that "jsmith1000" is plausibly an active user, whereas "jsmith(state:deleted)" is not. Hm.
The legal base for allowing this national rule in European law is Art. 6, 1c GDPR.
Generally, the rule is "Delete the data unless there's a law that requires you not to" — and the UK's implementation of the GDPR (the Data Protection Act 2018) makes various explicit exemptions for this.
1 month later, still no refund. Account is still scheduled to be auto-renewed, talk to CS and they're basically ignoring me at this point (I'm using text messaging support on a secondary number), so I issue a chargeback with my credit card...
1 month later, I get a refund, no email, no text message explaining the delay and now I have to deal with that or else they'll probably put my account in collections. /facepalm
If you use US companies for services in a European business, you are almost certainly going to have invoices every month that don't meet EU regulations at all and you just have to make it work.
I mean great if there are legal requirements for invoices, but who enforces them, how likely is enforcement, and what’s the end result for a US-based company with no physical presence in The Netherlands?
Being in the UK, we tend to work on a system where things are taken in context and you can defend such decisions. Maybe other tax regimes are more restrictive, but the British way is always that you can have a debate with authorities and usually they will see sense in your reasoning if you're not trying to defraud them.
https://twitter.com/i/status/1192182812121583617 The actual tweets probably explain it better than I (and the parent comment) can.
Since this predates threads, I can’t find all the tweets but this is one of them:
If they want to claim the expense they can find a competing company who issues an invoice.
You don't want to cascade that deletion to a record of credit card charges, but you also need to make sure that all queries respect that the user record might now be deleted - ie make it an outer-join.
It's far more robust to add an active/inactive field.
The longer an organization has been around, the more interconnected the database is, and the more consequent changes are needed to accommodate a core database change.
I would be surprised if many many organizations have some hack like this under the covers.
You make it sound like developers doing things the wrong way is the exception instead of the norm.
Good developers don't, but every place I've worked at has a few chunks of the software by people who didn't know or care enough to do things the right way.
Probably not. I bet that goes for so very many organizations. It is far easier to reinstate an accidentally (user) deleted account if it is merely marked as deleted than if it were truly deleted.
Now, how NYT is handling this is ... well, it's bad.
So I'm torn, because I think there may just be a major problem I've not yet grown my apps big enough to suffer. Anyone have thoughts either way?
That isn't what he said.
He said their inability to delete an account is due to poor design of their data schema by a well intentioned developer.
Of course "delete" meant rename it from username@custom.tld to usernameDELETED@custom.tld. He owns the entire domain (and has catchall) so he got the notification of the changed email to the new email address.
Now he has two EA accounts with all the games on both!
oh what a great service
With such a setup it would be possible to notice that NYT mail started coming to foo1000@... instead of foo@... after requesting account deletion. Username change could also be evident directly from the mail, or it was a trivial guess.
/edit I was wrong. It’s the user and email. Your theory would work
Why? My interpretation of "append 1000 to email" is that 1000 gets added to the very end, not the local part.
But I’d you use the altered email and the same password, everything is still there.
Pretty sure this goes against GDPR but I was totally unsuccessful at getting my account deleted.
Did you report this to your local privacy regulator (the ICO in the UK for example)? Not saying they'll do anything (I guess the "4% of global turnover" fines aren't enough to motivate them) but at least there's a record of it, and if anything else, a proof of how useless the whole regulation is.
/s
Unless someone figures out how to register a domain ending in ‘.com|disabled’ I’m not sure how someone would be able to access those accounts.
I believe it's easy to spoof emails from the .com|disabled domain. Receiving messages, I agree, seems harder. Maybe spoof an unencrypted DNS response at the right moment? No need to actually register a domain when DNS is spoofable.[1]
If you really need to use a hack like that to disable an email, consider adding some code to your email sending logic that skips such email addresses (and always use that logic). Otherwise clever hackers have a foothold to try their tricks against.
A lot of people are complaining about NYT approach, but perhaps their only fault - if one consider that not deleting for good an account is not an issue, and it seems to be a common practice in the industry - is to not use a transaction when disabling user accounts (disable email -> disable account), which is perhaps difficult with NoSQL setups?
DNS is a very old protocol that still has lots of problems and mitigations like DNSSEC are only partially deployed.
I mean imagine if HBO had deleted accounts during GoT and Westworld instead is suspending them. How many people had a 9 month subscription per year for years at a time?
Their system probably doesn't implement a function to delete account, or it is not easily discoverable in the UI, or it is known to be bugged. So the employee who did that thought that renaming an account was a good idea to make it appear "missing". A big numeric suffix is an obvious idea to avoid collisions with the existing and future accounts.
But when people are asked to choose a large number, numbers around 1000 and its multiples are chosen particularly often. So 999, 1000 and 1001 are very likely numbers to be picked "randomly". I don't find the reference on that, but I suppose we all have enough anecdotal evidence. Just recall what are the common port numbers of various programs. X*1000 + Y is a very common formula, where |Y| < 100.
Maybe this is what you're looking for?