edraferi

Senzing and Tamr are two options for this

How do you think Prefect compares to Airflow / Dagster?

Hey David - love what you’re doing with Retool. I think it’s a good solution for the front end piece of this challenge. Please correct me if I’m wrong, but doesn’t Retool focus on working with existing enterprise databases and APIs? If so then I think the missing piece is a DBaaS with great guardrails.

Access (and to a lesser extent, Excel) empower anyone to model their data and manipulate it without asking permission. The problem is they don’t scale well, there are no backups, they don’t show up in a data catalog anywhere, no integration with enterprise SSO, etc. They do benefit from file-focused solutions for these issues (eg the Access file lives on a network drive which is backed up) but they fall far short from what you expect from a production database.

So my challenge is: how can I let anyone in my organization create a new database, with all the backup/authentication/discovery/scalability out of the box, for them to connect to Retool?

I’m also very curious about this. I have several ideas for niche websites that I would love to build, but I want them to be sustainable without awful adware

Correct. Warren proposes to reinstate it:

https://techcrunch.com/2019/09/27/to-curb-lobbying-power-eli...

OTA isn’t a panacea though. You have to figure out ways to make it financially reasonable for people to blow the whistle on massive firms. The Waste, Fraud and Abuse laws might be a good template.

That's a perfect example of UI discoverability fails. The feature is super useful and perfectly hidden. Will use this all the time now, thanks!

> I couldn't figure out how to move my cursor around. It used to be perfect. I held my finger down, it let me move the cursor around in a word. Now you have to swipe the cursor to "grab" it and move it around under your finger. It feels like change for the sake of change when it worked perfectly before.

This 1000%. Working with text on iOS is such a pain now.

Maybe allow & incentivize people to add their own videos? Eg you get a month free for every high-quality flow video.

R for the ecosystem, Julia for the performance & future proofing.

I’ve found phonetic similarly algorithms like this to be very useful for data cleaning work, especially when your data generating process involves transliteration. That is, if your users know Language A but your data involves Language B, your users will record terms from Language B with highly varied spellings that nonetheless sound very similar when pronounced. Phonetic similarity algorithms help you cluster these spellings together by sound so you can map them to a canonical term from Language B.

Which is, of course, exactly what you want. These rules exist because charlatans bilked a lot of consumers out of a lot of money. The financial industry will do this consistently because the incentives are obvious. There has to be some external force slowing down the fraud.

BTW that was probably an illegal act by your employer. They gave you a clearly impossible task then fired you when you couldn’t complete it. That’s unreasonable.

You might want to contact an employment attorney in your state for a consultation. Your state Bar association can give you a referral and probably a free consultation. They will be particularly interested if this was the culmination of a pattern of bullying behavior (consequences vary by State) or if there’s some kind of protected class involved (eg you are over 40 and the manager wants a younger team).

It might seem like an unnecessary hassle, but legal action can help your remaining colleagues and possibly put some money in your pocket.

That’s a meaningless statement.

> Federally funded research and development centers (FFRDCs) are public-private partnerships which conduct research for the United States Government. They are administered in accordance with U.S. Code of Federal Regulations, Title 48, Part 35, Section 35.017 by universities and corporations. There are currently 42 recognized FFRDCs that are sponsored by the U.S. government.

https://en.wikipedia.org/wiki/Federally_funded_research_and_...

FFRDCs have a special legal ability to provide “neutral” input on technical matters. In theory Congress could hire Mitre or other FFRDCs to perform this service for them. That might take some tweaks to the law because FFRDC contracts seem to be aligned to specific executive agencies.

Personally, I’m a big believer in Federal in-sourcing combined with civil service reform. Using contractors for public interest positions, even contractors with special designations like FFRDCs, clouds the issue with contract management.

Of course, it’d be very difficult to design Civil Service reform properly and politically impossible to implement it, so creative contracting is probably the way to go for now.

I believe the distinction you’re looking for is Civil Service vs Appointed positions. The Civil Service is composed of employees who are hired and managed competitively (to the extent allowable by the unions and massive red tape...). Appointed positions are executive roles filled by the whim of the president, or the whim of someone the president delegates to. The most powerful appointed positions require Senate confirmation.

Almost everything winds up as a department under the executive branch, headed by a political appointee. But exceptions exist, like the Post Office and TVA.

Wasn’t much of this covered by the whole Semantic Web effort? They built a ton of stuff for self-describing data (RDF) that federated over multiple hosts by default (SPARQL), provided meaningful names (URIs everywhere) and multiple tools for giving all this semantic meaning (OWL, RDFS).

It stalled for many many reasons, including:

- XML is kind of awful

- Over-general solution without clear advantages for a specific use case

- Mis-aligned incentives

- easy to abuse open query capabilities

- Hard to use for both data publishers and consumers

I think that JSON Schema with incremental enhancement via JSON-LD is a more promising tech stack for another try. That would let you take advantage of the massive investments in the current API ecosystem while carrying forward the best parts of the old Semantic Web Effort.

Of course the incentives are still hard to sort out. Commercial entities WANT lock-in. They will Embrace/Extend/Extinguish anything they can because they need a moat to make money. Honestly anti-trust regulations might be needed.

Looking forward to seeing your project!

Sandstorm kind of still there. They discontinued the free tier for their hosted platform because they ran out of money. The founders went to work elsewhere but maintained the project on the side last time I checked.

I think it’s a shame, it’s a lovely concept. The Capability-based security alone is game-changing.

Details here: https://sandstorm.io/news/2018-08-27-discontinuing-free-plan

> To understand the current period, I recommend that you understand the workings of the 1935-45 period closely, which is the last time similar forces were at work to produce a similar dynamic.

Christ nobody wants to live in 1935-1945. That period set the All-time record for violent death

https://www.vox.com/2015/6/23/8832311/war-casualties-600-yea...

I evaluated several light markup languages a while back for use in technical contracts. Basically I was fed up dealing with MSFT Word’s formatting problems while iterating on complex content with internal and external lawyers.

Asciidoc definitely seemed like the best option. The biggest thing for me was the ability to do complex numbered headers.

Unfortunately the tooling wasn’t really good enough for my use case. I needed a clean way to move between Asciidoc and Word, and it wasn’t really possible. Pandoc claimed support for both Asciidoc and word, but it was only partial and couldn’t round-Trip the document. I considered developing & contributing the functionality, but I didn’t know the language the tool was written in and couldn’t justify investing more time in the experiment.

Maybe it’s time to review the tooling situation and take another crack at it...

What are you thinking about OFX/QFX? CSV is nice for breadth but some of the specialized formats are pretty nice for these kinds of tools.

+1 I got a lot of mileage out of the triple model when working with social media data. You just don’t know what data patterns you will find when you start looking, and need to support generic queries.

So just add a record separator at the end of each line?

God that game was so hard. Maybe I’m a wimp about rogue likes in general though. I enjoy stuff like XCOM where it’s a puzzle you can reason your way through and use save states to re-try when you realize your plan or your execution of the plan is insufficient for the task. There are random elements, but it’s mostly a puzzle.

FTL was just confusing, hopeless death for me.

+1

Interesting project. Who’s sponsoring this?

Looks cool, but has a contact form instead of a price sheet.

There’s a start up called TruePic that’s building a solution to this problem. Basically their platform helps you get trusted pictures from users.

https://truepic.com/

Sorry about that. People should use these kinds of questions as an opportunity to educate, not brag/exclude/shame.

Relevant XKCD: https://www.xkcd.com/1053/

Because Paul Graham, who founded Y Combinator and by extension Hacker News, loves Lisp. He used it as the primary language at Viaweb (basically late 90s Shopify) and viewed the language as a key enabled of their success.

Paul’s blog posts on the subject are useful background: http://paulgraham.com/lisp.html

You should probably start with this one: http://paulgraham.com/avg.html