apwheele

I expect `python hello_world.py`, but if people are confused I just nudge them to what I expect. It is not meant to be a trick question.

If people do not have local setup, I just have them write out in text editor and walk through the steps. Maybe not 75% fail rate, but more like 50% of people fail this step in the tech round.

Even before the coding assistants I was not happy with homework. It was not discriminatory, everyone looked the same, https://andrewpwheeler.com/2022/03/24/i-have-no-clue-how-to-....

I do simple questions similar to the quoted LinkedIn post. (So less leetcode and more "do you know how to code anything".) While I can agree it is harder under stress, what is the alternative to knowing if someone can code at all? (My pass rate is similar to the quoted LinkedIn post.)

I have done the entry interview as well. For example have had very good conversations with managers who were applying for senior IC roles. They then go and fail the tech interview basic "can you do write and execute a python hello world script from the command line".

Client side it is aggregations of the data, (so yes you could run out of memory). DuckDB does not load all of the data client side.

For a newer list I would add the mapbox API as well.

So I work in data analytics, not so much web-mapping. For those applications, IMO local solutions, like ESRI, are good options if you are limited to addresses in the US, https://crimede-coder.com/blogposts/2024/LocalGeocoding.

Googles TOS was that you can't even cache the results, https://cloud.google.com/maps-platform/terms. So saving to a database and doing analysis of your data is not allowed AFAICT.

Care to share the contexts in which someone needs a zero-shot model for time series? I have just never come across one in which you don't have some historical data to fit a model and go from there.

I feel the same way, and a good follow up book about more general communication is Trees, maps and theorems by Doumont, https://andrewpwheeler.com/2016/12/05/review-of-trees-maps-a...

That said, I think it is possible my refusal to do cheeky ppt slides with smart art and fill them with graphs of real data instead has stunted my career growth into management.

IMO if doing this, you should avoid text in the charts entirely (as the title can sometimes I think lead the models astray, such as the clustering title I think will bias it to find clusters even if none exist). Presuming you are the one making the chart and not just prompting with another image.

I believe the text in the image will be more prone to misinterpretation that direct text in the prompt anyway, https://andrewpwheeler.com/2024/07/16/using-genai-to-describ...

Wayback url, https://web.archive.org/web/20241118131800/https://www.newso...

My companies looks similarish to the recent screenshot, but it is a hellscape of a billion options and poor search functionality. To the extent I just need to ask a person the right link or tree search whenever I need to actually use it.

I don't envy developers who need to work on this, but IMO the best systems I have worked with have a very shallow tree and then a "human will work out the appropriate team to route to".

A tell for fake firms in my local newspaper is they ask for a snail mail resume. These appear to me to be more like shell companies submitting multiple H1Bs as far as I can tell though, not legit firms saying they cannot hire any US.

I agree understanding KM is a very good place to start survival analysis. Many examples in my business I have for KM the censoring is due to certain events taking along time (auditing healthcare claims) to resolve.

When I first learned survival analysis, my professor had me construct life-tables, and then learned KM. You can often do quite a bit with discrete time tables, so if you have data:

    ID TimeRange Outcome
     A   4          1
     B   3          0

You can then explode the data into the form:

    ID Time Outcome
     A   1     0
     A   2     0
     A   3     0
     A   4     1
     B   1     0
     B   2     0
     B   3     0

If you groupby this table and get the numerator/denominator, that is what you need to calculate the life-table, and the discrete version of the KM plot.

Understanding that also allows you to use more typical binary regression or machine learning models, and then you just calculate the cumulative hazard from the predictions afterwards, https://andrewpwheeler.com/2020/09/26/discrete-time-survival....

Care to elaborate on this? So this post does not save the resulting weight, so you don't use that in any subsequent calculations. You would just treat the result as a simple random sample. So it is unclear why this critique matters.

Very nice, another pro-tip for folks is that you can set the weights to get approximate stratified sampling. So say group A had 100,000 rows, and group B had 10,000 rows, and you wanted each in the resulting to have approximately the same proportion. You would set the weight for each A row to be 1/100,000 and for B to be 1/10,000.

If you want exact counts I think you would need to do RANK and PARTITION BY.

I'm sorry but this model is wild and highly misleading. Based on one variable and 7 observations, days since you have started, you have fit a model with `2 + 128 + 64^2` parameters. It happens that a straight line is fine for this data, so it does OK approximating the two observations that are in the test split.

I was curious to see what would happen out of sample, I figured it would just be a straight line (and it is, despite the parameters being quite complex, print out `model.weights` to check them out!)

    outS = [[365],[400],[450],[500],[2*365],[3*365],[4*365]]
    outY = model.predict(scaler_X.transform(outS))
    outY = scaler_y.inverse_transform(outY)

Negative if you keep at it long enough.

So have not dealt with McKinsey specifically, but other MBA/Consultant types I feel like they treat software engineering like we are ditch diggers -- they just request things and expect it to magically happen.

So most of the requests are insipid, some are impossible, and they have no desire to spend effort to understand either.

So the message on my desktop is below where I can see, and ditto when I preview in phone (either orientation). So still a problem. IMO I would just make the input box smaller.

And maybe do a legend for the pie chart, so immediately clearer what I am looking at instead of making me hover.

I was aware of that, but I would still be interested to know the actual costs. I do not think "it is ok to blog about potential fraud in scientific publications because GoFundMe will pay for my lawyers" is a good assumption for most individuals.

I see people saying it is a win for open science, and I just look at that Court Listener page and it still fills me with angst.

While a favorable ruling for the Data Colada defendants seemed incredibly likely in the end (and of course being dismissed pre-trial before discovery is a good thing). But it still took over a year, and consisted of several pre-trial hearings with multiple back and forths between them. If I ctrl-f https://storage.courtlistener.com/recap/gov.uscourts.mad.259... I see Pyle (their lawyer) listed 11 times. All the motions and responses are a non-trivial amount of work.

Do folks have a sense of how much the costs would have been just to get dismissed for DC? Seems to be definitely over 10k (50k?) Or am I overestimating.

In the same page you quote, https://crimede-coder.com/services/PredictiveAnalytics, the next sentence states "CRIME De-Coder has developed methodology to make predictive analytics more fair and racially equitable (Circo & Wheeler, 2022; Wheeler, 2020)."

In terms of turn inward, check out the work I have done on early intervention systems (to identify problematic officers), https://crimede-coder.com/blogposts/2024/EIS.

I have done quite a few evaluations of prosocial interventions in my career as well, my work is not solely focused on proactive policing, https://scholar.google.com/citations?user=iNNqtgwAAAAJ&hl=en

The examples are crime analysis focused, you can see the final chapter end-project example here, https://github.com/apwheele/CrimeBook. But yes the fundamentals are the same for most any "data analyst job".

If you look at the preface section "What this book is not", I discuss that point:

> This book is aimed to get you started writing code and applying it to real tasks crime analysts need to conduct. I use realistic examples that a crime analyst may be interested in conducting, such as sending automated emails, making year-to-date tables, and creating line charts. But I do not discuss in detail things like the Poisson distribution for analyzing crime rates or why hotspot analyses is important.

Even with secrets if the CICD machine can talk to the internet, you could just broadcast the secrets to wherever (assuming you can edit the yaml and trigger the CICD workflow).

I was thinking maybe a better approach instead of CICD SSH into prod machine is to have the prod machine just listen to changes in git.

This seems to come from a place that "learning virtual environments is too much work". I do not agree with that premise. Having a lab where you introduce environments and the command line (which you can do in a single class for the types of students Paul is talking about) is not a big deal and to me is a good use of time.

This is a good take on that admin like identifying dupes is a feature and can be grating for newcomers.

But I don't think their is a practical upper bound on good questions though, so the reduction in total questions and answers in those graphs is definitely not good.

I have always thought the scale itself is very hard to maintain. I thought the different sites had a good balance but it is hard work. Mods and superusers should get compensation.

This was my thought as well.

I don't like showing the scatterplots in these examples, as "correlation" I think is more associated with the correlation coefficient than the more generic independence that the author means in this scenario. E.g. a U shape in the scatterplot may have a zero correlation coefficient but is not conditionally independent.

They had a similar article for some of the warehouses in the states and I showed the rates are what you would expect using OSHA injury data, https://andrewpwheeler.com/2022/11/05/injury-rates-at-amazon...

This example could be all javascript (and I try to do that in some scenarios). Most of the other demo's on my site are just javascript/D3 or other libraries even with some slightly more advanced metrics, https://crimede-coder.com/demos.

There are some cases in which I do want python numerical libraries though. Say I wanted to do a forecast ARIMA model based on arbitrary combined inputs for the graphs, or do different types of spatial clustering of the geographic data. Those would be cases pyscript would make sense.

The public dashboards yes IMO easier to not have to worry about server.

Yes this is part of the price of running it client side instead of having a server do all the data manipulation. It is pretty much a wash for RAM for me vs running locally (local is 200-300 for python + 100-200 for chrome for just that web-page).

For folks interested in seeing a usecase, I have a panel dashboard of Dallas crime statistics at:

https://crimede-coder.com/graphs/Dallas_Dashboard

Startup takes around a minute (which includes installing various python libraries + loading an external 5mb dataset). But once up and running it is quite responsive. (And works fine on my iphone.)

Many businesses I work with use Tableau free versions, and just manually update the data (since the free doesn't allow direct connections to a datastore). This is a free alternative (I use github actions to build a zipped up csv file that is pulled into the environment).

Yes it should be a "+", thank you.

Just because marginal cost for software can be close to zero in some circumstances does not make demand curves meaningless!