brootstrap

I feel like they should be able to pull off a nice looking pokemon game. Think Breath of the Wild but pokemon! This looks pretty close to that no?

After years of working with AWS this is not surprising. You can have developers with minimal sys admin / dev ops building instances and running jobs. Back in early days of my startup I recall my team launching a big model run and leaving for weekend. We can back on monday to a 10k bill and thousands of compute hours... That did not go well, 10k on a few days of compute for a small startup haha! Still to this day we've got tons of EC2 instances lying around racking up charges. We dont have the devops support to build proper auto-scaling, making things really fit AWS. We just needed a server to run 24/7 and this was easy.

Can only imagine the insanity that goes on at huge Pinterest level scale. How many EC2 instances do they have? Thousands? Hundreds of thousands?

haha. if i was walking outside and ran into wolfram like this, i'd fucking bookdrop his stupid ass laptop and watch him cry and freakout. I work in town with wolfram HQ, have had relationships with folks who work very closely with wolfram. As you could imagine he is a fucking nut case haha.

all of my experience with McKinsey has been an utter waste. Worked with multiple contractors on multiple backend database projects. They work like a walking ad for AWS services but dont actually know shit. We have a 10+TB dataset (growing by gigs each day) that is spread across a few postgres servers. These servers are decent but foreseen as bottle neck in coming years.

This guys suggestion is to put our data in redis b/c it's fast and in memory. Looking up hosted REDIS solutions, (redislabs, we use for other smaller cache redis things). To put 1 TB of data in standard redis server would be ~50k/month :p

100% of my experience with McKinsey has been trash. Just started working with new company and they wanted us to talk with these guys. We are reviewing some database systems and this guy is a literal joker. His suggestion to speeding up our database "well we can use REDIS, REDIS is fast because it's in-memory". So we want to put how many TB of data into REDIS? We are talking about tons of historic data that never changes. And we are going to load it ALL into REDIS because it's fast. These McKinsey folks spend months talking about the data lake and it turns out data lake just means a bunch of shit on s3.

It's like if you paid someone $50,000 to help you find a good car. Client wants a fast car. Oh you want a fast car? Buy a ferrari they have big engines and can generate tons of horsepower. Job done, can i collect my 50k now?

Mind you this guys linkedIn and resume read like a CEO wet dream. AI and machine learning data science all over...

We spent months dilly dallying with different tech before they just left cold turkey , havent heard from em since. And our 20% complete projects experimenting with all kinds of DB tech have done nothing.

If people are using a real database, likely there is just a boolean flag for that record getting flipped to True/False to signify deletion. Unless you use mongo, then your shit is probably gone already and you dont even know it.

haha. user.is_deleted = True...

A lesson on how to delete user data but still keep it. RE the headline, i'm not sure. "News flash, mega tech corp thats business model is based on accumulating data has been accumulating lots of user data."

I didnt get those vibes either. Wasnt it the yacht guy himself talking about his dependence on instagram? Anyways, props to instagram guy for getting his $$$ when he can. Dude is right, only a matter of time before instagram sucks and there is some new hotness.

This whole social media influencer thing is kind of funny. Of course I think it is trash and that likely some large percentage of followers are bots. However, it does have some degree of power in certain circles. Just yesterday on sports radio these talk show hosts (been working for 20+ years in journalism) are quoting people on facebook/twitter etc because they have 70k followers. The host literally said, I dont know who this guy is but he has 70k followers and knows Manny Machado's dad.

Valid point. I have to give some updoots for VIM, (or emacs too i guess). The visual editing gives you such an advantage, i'm talking just about coding. Most of my co-workers come from science background and think VI is a joke from that crazy professor in college. I forced myself to really learn VI one summer in 2013. Since then i've been flying around my text editor.

I'm not lying, co-workers think you are a god damn wizard. I try not to be a doosh, but I have one co-worker who loves VIM but doesnt really use it (insert mode ftw). Whenever I can i try to subtly drop hints. "Instead of hitting the arrows keys over and then backspacing, you can type cw to change the word" Just little things like that. Once you realize that vim is modular and chainable, you really start to take off!

your name is also duderific so you must be a fantastic duderino to hang out with! Who wouldnt want to work with such a dudeson.

If i was a wizard I would be smoking that enchanted nug. Pack up a bowl with magic and use your wand to perfectly corner it. Magic!

Turns out i've already beaten the LOTR challenge mode in a sense. Rainy saturday, all day LOTR marathon on TBS, sack a nugs, plenty of grub to eat while watching the movie all day

POTATOES... Biggest complaint about the post is lack of boiled, mashed, and or stewed potatoes.

I know people love the wireless thing. But if you want some nice over ear cans, and dont mind a wire. Sennheiser HD 280 i think , should be around 100 bucks. Not sure who Jabra is but if it's some kind of fancy startup making headphones. No thanks. I will stick to people who do actual audio work to make my headphones please.

hyperlocal is a joke, sorry to say it. Just because you can downscale your grid to a very small number doesnt mean good data. It just means you can do statistics and mathy stuff to interpolate. Everyone forecasts better in stable times such as a strong high pressure ridge. The tricky part is the edge areas between air masses, the transition zones. This is where all the juicy weather actually happens and it is the hardest to model

Yes this is truth to some degree. However, you dont know what you are getting with an app. If you go to read the official NWS page, the forecasts are human curated by a pro. Apps likely are just taking model output directly and feeding it to you as gospel.

Your last statement is the so true. An amateur forecaster can spend ten minutes looking at the some big picture (synoptic scale) weather and know regional trends (this part of country is dry, these guys are wet, humid etc). This would be a pressure & wind map, showing location of high/lows, maybe some 850mb moisture charts too... But you are painting broad strokes. All of these dumbos on HN want to know exactly when the rain is going to hit your exact location, or you have to be within 3 degrees C modeling the surface temperature of my street so i can either ride my bike or not. We just cant do that with the current science & computational landscape!

After you have the synoptic scale picture, then you zoom in on your target area and get specific if you want to. This local expertise is where you NWS office comes into play. You want the local forecast from these guys!

Howdy from across the pond! To be honest, I have almost no experience forecasting for Europe so sadly I cant help too much with the specifics. You are right predicting rainfall at small scale is very hard.

I know that here in the midwest, we never have any days where the forecast at 5am says 'all sunny and zero chance of rain', and then randomly a huge thunderstorm pops up. The atmopshere just doesnt work that way. People get upset about probabilities of rainfall and all that. Just think of the model perspective. The model sees square grid cells that are many KM apart. If the storm shifts from one grid cell to the next one, not that big of a deal in the grand scheme of things. However if you live in that grid cell, your forecast just changed from getting dumped on to clear skies (or vice versa).

The system is complex beyond our comprehension. If we really wanted to model weather and climate better we need to know more things. Things like how much cow farts are there? I would be happy to build a system to ingest cow fart obs and feed them into the numerical models. FPS farts per sec yo

Just a little knowledge to drop, as a bachelor degree meteorologist and also coder / HN'er.

Dont use your built in phone app for forecasts. Dont expect hour by hour precision (maybe for the next few hours sure, but not 2,3,4,5 days out). Trust nobody expect the trained professionals at NWS. These people look at ALL the models, they watch things like a hawk. I work with some of these nerds and it's crazy. They are like 'oh jimbob did you see the latest GFS kicking out the low up into canada? oh yeah doug that is crazy, but the euro says blah blah and NAM says blahblah... Their job is to literally make a forecast, for the same area, every day. Some people do it for years and years! Think of the experience, wisdom, little tricks and tips etc. Just go to weather.gov, look at it for 2 minutes. Read the foreacst discussion if you want to get your details.

Validating the wx is extremely hard. For instance, if you make a forecast, what do you use as the 'truth'? A weather station? Trust me, I've worked with plenty of station data, and it is trash. So now you are comparing a weather forecast against some suspect trash validation set. Nobody knows!

The fact is, predictions will keep getting gradually better, more high res. But the fact our atmosphere & earth system is insanely complex. The system is chaotic so tiny small changes in each model run produce different results... We have code to represent some basic physics but still dont understand many of the important things. We still dont know exactly the rain process & snow formation process. We are trying to pinpoint the exact moment when dust & water vapor magically form into a cloud droplet so we can better describe the rain process. THEN we have to translate all of this insane math and physics into code.

oh man - you want to know something funny. At my startup we had a guy hacker in a random weather generator to use in our models. I was like WTF Man!

The numbers you are getting are not random. Sure the models are chaotic and non-linear, but still it's giving you hints at potential patterns, especially at long range time scales. People need to understand that wx forecasts, especially 3+ days out, should be read as guidelines for what is LIKELY to happen. Say that GFS predicts two inches of rain for your city over the next week as a wave comes through. What if the next model run shifts that blob of rainfall 25miles north? In the model terms, global terms, 25miles is nothing. But for you that is the difference between dry & soaked , and you think the model sucks.

They also dont even reference any USA models in the article. The GFS was also showing this big cold snap, not just the euro!

Hmmm. If you are needing down to the minute accurate timing of when rain is going to start & stop, sure nobody can predict that, for every location in the USA, all the time. I have to question where you are getting your weather forecast information??? I am a bachelor degree meteorologist, I dont use any apps that try to be 'hyper local' (omg i hate that word, it is the weather version of AI/ML hype). Just go to the government site www.weather.gov

We can't predict exactly when it will rain, but we can get daily trends pretty good for 3-5 days out. This means you are five days ahead of the weather. You cant know "it will rain from 5:00am to 9:30am on february 4th, 2019". But you can know if we have a synoptic scale system that is moving through your area on that day, and know if there is a probability of rain.

Where do you work everyday outside? i would be happy to give you a 5 - 10 day forecast right now actually and can tell you it will be good. I am supposed to be writing some tests and documentation but i can drop some WX knowledge if you need.

Ah, no need to give me all the details good sir. Appreciate the post. Also yeah, sounds about right with the pre-processing. That is always the case!

appreciate the write up but is this even AI? It seems like he just did some analysis & stats on observed data, and came up with a simple model.

Interesting and good to learn about this real problem. The show had me really on edge (like all black mirror content) until they kind of broke the 4th (5th wall?) with the netflix thing at the end. Watching this guy talk to his therapist because he was being controlled by netflix was friggen hilarious to me...

I think i have felt this way at time, usually when really high. The whole LSD trip with colin in particular.. I liked that rant, the multiple realities, the time is a flat circle thing.

The movie hit pretty hard, good on black mirror guys for pushing boundaries of film as we know. Even some folks who dont usually get into black mirror were like, well i want to retry it and pick the different cereal to see what happens!

For some people (not fucking pats fans though), this is a once in a lifetime chance to see THEIR team play in the biggest game . So yeah, people travel, migrate. For some people it feels like a religious journey. All of your kin are migrating across country for this one event... Just being in the city is cool, you will be seeing rams / patriots jerseys all over Atlanta for the whole week!

I think you bring up a good point. People imprint their own perception of things to this AI game. The bot is probably working like other deep learning machine learning things. It runs some models or whatever and tries to minimize error. Not surprising really that the computer thinks differently than humans. A worker rush sounds hilarious to us but hey it can work.

Ok given what we've seen from headlines recently, this actually makes sense. My grandma is old and I read older people share fake news more then youngins. My grandma (and older aunts/uncles) are the only people who forward emails. I wouldnt be surprised if lots of fake news articles were forwarded around networks of older folks.