HNHacker News
TopNewBestAskShowJobs

apwheele

1,910 karma · joined November 10, 2021

Data scientist. Former academic in criminal justice field.

Personal blog at https://andrewpwheeler.com/

Consulting at https://crimede-coder.com/

Large Language Models for Mortals: A Practical Guide for Analysts (book), https://crimede-coder.com/blogposts/2026/LLMsForMortals

submissionscomments
apwheele··on Ask HN: How to Price a Product
In https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2020.360..., assuming the demand curve slopes downward, a quite reasonable guess as to the optimal point on the demand curve is ([Max willing to pay] - [marginal cost])/2.

So if doing the market research here in this post, #1 in your above list is the main thing. You want to find the reasonable upper bound before the demand goes to 0 on the price curve.

For one time sales, like in this scenario, it is a fairly simple way to price things.

apwheele··on Cartographic decisions on a leaflet hotspot of crime map [video]
The map to view directly is here, https://crimede-coder.com/graphs/DurhamHotspots

I made a video to discuss several of the cartographic decisions in making an interactive hotspot map of crime in Durham, NC. Discussion includes hover functionality (so you can see underneath the area), popups vs tooltips, and clustermarkers in leaflet for dense point patterns and being able to interact with overlapping points.

apwheele··on Show HN: Play with an interactive heatmap of SF crime (and other cities)
I am not a fan of the kernel density estimators with a continuous color ramp (at least this map the KDE does not recalculate on zoom, that is pretty much worthless).

Currently the UI on the left returns all empty images for me, https://www.dropbox.com/scl/fi/re068bj1vnaitsoh59x27/NoImage...

I rather enjoy using either points directly (and for dense point maps use leaflet's cluster markers), or actual outlined areas (e.g. using DBSCAN). For a few examples I have put together

- Dallas crime dashboard (using wasm + python), https://crimede-coder.com/graphs/Dallas_Dashboard

- Durham NC hotspots (leaflet), https://crimede-coder.com/graphs/DurhamHotspots

apwheele··on Police Departments should share crime data
This is too broad IMO. It is on the edge of LEO sensitive info, so even if you did a FOIA request I think you would be shot down. I don't think there is any chance PDs will voluntarily post this.

You can search your local PD, try to find a contact email for their analysis or community police department and send an email to ask. (Not great chances that works either, I would suggest trying to limit your ask in a way that makes it much less likely to contain sensitive info.)

apwheele··on Poisson designs and minimum detectable effects
The quoted statement is IMO fine -- standard error as a term is only used to refer to the error around an estimate, not the marginal distribution.

Now to the target audience, it was criminologists monitoring counts of things over time. But that monitoring counts of thing over time is applicable to other areas obviously, such as A/B testing for rare events.

apwheele··on Show HN: Tea-tasting, a Python package for the statistical analysis of A/B tests
For statsmodels for the methods I am familiar with you can pass in frequency weights, https://www.statsmodels.org/stable/generated/statsmodels.gen...

So that will be a bit different than r style formula's using cbind, but yes if you only have a few categories of data using weights makes sense. (Even many of sklearn's functions allow you to pass in weights.)

I have not worked out closed form for logit regression, but for Poisson regression you can get closed form for the incident rate ratio, https://andrewpwheeler.com/2024/03/18/poisson-designs-and-mi.... So no need to use maximum likelihood at all in that scenario.

apwheele··on Why is it so hard to share links on LinkedIn?
So I am tiny fish, https://www.linkedin.com/company/crime-de-coder/, so not quite sure what typical impressions look like for the big influencers (currently my company webpage has 2700 followers). LinkedIn lets you grab your stats for posts over the past year -- have only recently heard the thing about urls, I pretty much always shared posts with URLs. (My company webpage for along time was not indexed at all on Bing, but my LinkedIn posts did come up.)

Here are my stats aggregated to domain for impressions for all domains I posted 2+ over the past year. I suspected youtube links are not promoted, and this confirms this. Also the local Raleigh newspaper is not doing so hot. But otherwise my NO URL posts don't do any better offhand than the linked ones.

    | domain                                |   count |   mean |   std |   min |   25% |   50% |   75% |   max |
    |:--------------------------------------|--------:|-------:|------:|------:|------:|------:|------:|------:|
    | crimede-coder.com                     |      61 |   1304 |  1261 |   345 |   625 |   854 |  1380 |  7179 |
    | andrewpwheeler.com                    |      53 |   1309 |   820 |   481 |   813 |  1074 |  1501 |  4463 |
    | NO URL                                |      13 |    937 |   489 |   453 |   692 |   850 |   909 |  2413 |
    | www.tandfonline.com                   |       6 |   1655 |   682 |   967 |  1119 |  1572 |  1960 |  2764 |
    | journals.sagepub.com                  |       6 |   1613 |   983 |   669 |   839 |  1363 |  2207 |  3134 |
    | link.springer.com                     |       5 |   3029 |  2810 |  1478 |  1722 |  1753 |  2154 |  8038 |
    | github.com                            |       5 |    873 |   170 |   658 |   838 |   864 |   874 |  1134 |
    | www.newsobserver.com                  |       5 |    496 |   144 |   363 |   365 |   459 |   626 |   670 |
    | www.crimrxiv.com                      |       4 |   1990 |  1678 |   489 |   585 |  1871 |  3275 |  3730 |
    | onlinelibrary.wiley.com               |       4 |   1851 |  1258 |   817 |  1276 |  1452 |  2028 |  3684 |
    | www.carolinascrimeanalysis.org        |       4 |    988 |   198 |   799 |   850 |   956 |  1094 |  1241 |
    | www.youtube.com                       |       3 |    457 |   160 |   281 |   388 |   496 |   545 |   595 |
    | theconversation.com                   |       2 |   2308 |  1151 |  1494 |  1901 |  2308 |  2715 |  3123 |
    | crimesciencejournal.biomedcentral.com |       2 |   2158 |   987 |  1460 |  1809 |  2158 |  2507 |  2856 |
    | papers.ssrn.com                       |       2 |   1719 |   387 |  1445 |  1582 |  1719 |  1856 |  1993 |
    | alexchohlaswood.com                   |       2 |   1505 |   585 |  1091 |  1298 |  1505 |  1712 |  1919 |
    | www.jratcliffe.net                    |       2 |   1349 |   552 |   958 |  1153 |  1349 |  1544 |  1740 |
    | osf.io                                |       2 |   1138 |   465 |   809 |   973 |  1138 |  1303 |  1468 |
    | academic.oup.com                      |       2 |    874 |   379 |   606 |   740 |   874 |  1008 |  1143 |
    | asc41.org                             |       2 |    841 |   665 |   371 |   606 |   841 |  1076 |  1312 |
    | www.amazon.com                        |       2 |    803 |   151 |   696 |   749 |   803 |   856 |   910 |
    | americansebp.org                      |       2 |    798 |   627 |   354 |   576 |   798 |  1020 |  1242 |

So no experiment, but some hard numbers at least!
apwheele··on Show HN: Briefer – Multiplayer notebooks with schedules, SQL, and built-in LLMs
I have a question I ask people I interview now for data science jobs -- can you run a python hello world from the command line? A non-trivial number of people who say they have multiple years of data science + python experience do not know how to do this!

People need to know how to write functions to write professional software, not just put a soup together of notebook cells.

Disagree with Matt even that they won't like this code -- it is just learning basic stuff like file structure and python environments, see how I approach it in the first two chapters running stuff from REPL here https://crimede-coder.com/blogposts/2023/EarlyReleasePython.

apwheele··on Database Design for Google Calendar: A Tutorial
Data scientist, and I have had a few examples of seemingly simple "how would you build a schema" job interview questions that I had a difficult time with on the spot.

So last one I remember was how would you build a product table with coupons. Ok, so two tables right, no big deal. Well, we are going to need to keep a history right? So now I need to update and have datetimes for different products and coupons. And now I should think about how to do indexes on the tables, and gosh my join to get the discounted price is that a good way to do that? Most coupons only allow a person to use them once, how the hell am I going to implement that?

They probably just wanted the simple product + coupon table, but let me spin on it for quite a while like a madman.

apwheele··on Ask HN: Trouble learning things I view as solutions looking for a problem
When you get asked questions IMO I would just swap out your experience for equivalent, e.g. do you have Jenkins experience - no, but I have created my own automated CICD processes to build test and push. (Part of the reason to do CICD stuff is to have another computer do that stuff in an org with many people contributing, but it is the same principle as you built locally.)

For cloud I would swap out discussing rsync and SFTP in much the same manner, although that is slightly more of a stretch, most people asking those questions either won't know better or will know idiosyncratic cloud stuff can be learned if you are at that level.

apwheele··on What we should do about the mentally ill homeless
IMO the long acting drugs + more intensive caseworker intervention is the most feasible way for the very difficult to work with individuals. (Long acting is good for opioids as well, https://healthandjusticejournal.biomedcentral.com/articles/1...).

So there is a mixture of types of homeless -- some of the examples Scott gets into for "draconian" are for what I would call transient/short-term -- those folks some welfare type intervention (housing vouchers, cash) to smooth over their instability is all they need. Those are not the homeless people are complaining about!

The "pollution" homeless though Scott is mostly talking about are the people who cause public disorder. And those are very hard to deal with. So everything will be difficult, but I think caseworkers being able to help them as much as possible is IMO the best approach I've seen work in some cases.

apwheele··on Show HN: Pdfscale
I have not used this specific tool, but have used https://community.coherentpdf.com/ for the cropping/rescaling. And in general "printing" (posters) is what I have used it for (books make sense though as well).

Depending on the site and what you are doing it may auto-rescale, I try to make it exact though just to prevent any nonsense on their end (and in Adobe PDF you can print tile to check it out as well in actual size).

apwheele··on ShotSpotter does not reduce gun violence or increase clearance rates
So showing up 1-2 minutes earlier does indeed increase survival rates. For instances in which someone is actually shot, I estimate that -2 minutes = 1% increase in survival, https://link.springer.com/article/10.1007/s12061-020-09362-3. So in that case, ShotSpotter will save a life around every 100 shooting victims it covers.

Part of the rub is that people being hit are not all that common. (It may be false positives, or just no recovered shells on scene, but many of the ShotSpotter calls result in a nothingburger.) So what has happened in some cities is that ShotSpotter has increased workload in going to calls, so may have had negative spillover in that regards.

apwheele··on Ask HN: How do you structure your shared finances with your spouse/partner
High yields are 5% now -- I get who cares about 2k now, personally I wish I did the high yield earlier. I did somewhat similar to OP for maybe 10 years and lost out on low 5 figures when I go back and calculate interest lost.

I am familiar with the perks for BoA and they are not worth losing out on high yield interest at this range. If the reason is for safe deposit though (as the OP said) it is very much not worth it. (I can't find on google Fidelity perks easily, mortgage is the only one that could plausibly make sense, but I am skeptical of that.)

So my high yield is liquid, I can do large purchases on credit card and pay it off with high yield. It did only take a few hours to create the account for what it is worth, so again IMO worth it over parking a large sum of money in a checking.

apwheele··on Ask HN: How do you structure your shared finances with your spouse/partner
I imagine you have gotten this advice before, but IMO a high yield savings probably makes sense. Say keep 10k in checking for bills, and then the other 40k in high yield. It will more than easily pay for the safe deposit box!

For myself we have joint everything, and then purchases over a certain threshold we need each others permission.

apwheele··on Ask HN: Machine learning engineers, what do you do at work?
I am speaking more generally of the industry. You can look at my profile and follow the bread-trail to see whom I work for. It is not insurance, we are a client of state systems, and I work to do the opposite of what you suggest (identify fraud/waste/abuse in insurance claims).

I do not know the chances that any individual provider uses a system to identify what codes to tack onto your bill.

apwheele··on Ask HN: Machine learning engineers, what do you do at work?
The scale of health insurance claims is incredible, my company has a process that simply identifies when car insurance should pay the bill instead of medicaid/medicare post traffic accident (subrogation). Seems a minor thing right? We process right at 1 billion a year in related claims (and I don't know our US market share, maybe like 10-20%).

I am guessing every datascientist that works for a BlueCross BlueShield at individual states deals with processes that touch multiple-million dollars of claims.

We even now have various dueling systems -- one company has a model to tack on more diagnoses to push up the bill, another has a process to identify that upcoding. One company has models to auto accept/deny claims, another has an automatic prior authorization process to try to usurp that later denial stage, etc.

apwheele··on It looks a lot like VMware just lost a 24,000-VM customer
Similar to pmx's point, an implicit assumption in this (which is unlikely to be true in many scenario's) is that the there is a negligible spread in the distribution of revenue per customer. Imagine if you have one whale customer and a bunch of small fish, if you lose the whale in the change you won't maintain the same revenue.

Some scenario's this assumption is reasonable though (say Netflix subscribers).

apwheele··on Databases for data scientists
Yep, https://duckdb.org/docs/extensions/azure
apwheele··on A useful productivity measure?
This is not inconsistent with How to Measure Anything IMO (I like that book as well). The biggest issue to me is that he does not define actual follow ups on ROI -- it is all estimated in this framework. So it is all good to define how to prioritize, it is not helpful though retrospectively to see if people are making good estimates.

My work very rarely is a nicely isolated new thing -- I am building something on-top of an already existing product. In these scenarios ROI is more difficult -- you need some sort of counterfactual profit absent the upgrade. Most people just take total profit for some line, which is very misleading in the case of incremental improvements.

apwheele··on Bollards: Why and What
Bollards are also good ideas to prevent intentional terrorist acts of driving cars into pedestrian areas, https://www.nbcnews.com/slideshow/terrorist-truck-attack-sho....
apwheele··on Flock Safety is the biggest player in a city-by-city scramble for surveillance
OP for sharing this on hackernews (and I talked to Tyler Dukes a bit about this). I think ALPR's are good investments, but having more rigorous standards in place for when people can do searches is necessary. There are rules for when you can run a criminal history background in states I am familiar with (that are policy set, so less rigorous than a warrant), that I think should be applied the same for searching license plates without too much friction for law enforcement.

I think Flock has a good product (and ditto I think Dolt is neat!) But that said, this 10% metric is so ridiculous it rises to the level I need to make a comment. Imagine I did something to decrease crime by 10% in two cities, and then went and made a claim like "I decreased crime 10% in the US" -- this is Flock's claim. (The study to get the 10% clearance estimate is crazy bad as well, but this 10% of solved crimes in US is such a bizarro projection to the entire US it is inarguable as to its absurdity).

apwheele··on Ask HN: What tool(s) do you use to code review and deploy SQL scripts?
We are a python shop (end to end machine learning, so we do all of our data pipelines ourselves).

So ours is pytest that builds up a fake database + tables to unit test the scripts. And otherwise it is the normal review process for pull requests.

So ours is daisy chained as well, but I am quite happy with it.

apwheele··on US city-country mortality gap widens
I think this is right. For counterpoint though, there will of course be variance, but an opposite hypothesis is a larger proportion have labor jobs (men moreso than women).

Grew up in rural PA (mostly farming, some factory work). They exact a significant toll (part of the reason opioid dependence grew IMO).

But most of the metrics I have been seeing in terms of death rates show it is both for men and women, so I don't think labor jobs can explain it.

apwheele··on Ask HN: Freelancer? Seeking freelancer? (April 2024)
SEEKING WORK | Remote | Raleigh, NC, USA

PhD (in criminal justice), and with my background mostly focus on mathematical modeling in the public sector. This includes program analysis (I did X and want to see if it made Y go down), forecasting/machine-learning (I have won several prediction competitions by NASA/NIJ), and optimal allocation (integer linear programming, GIS, and network algorithms).

Doing this work you pick up a bit of over tech along the way, so I do have competence in front end and back end. So if you wanted a javascript app that queries a backend database instead of your current Tableau dashboard, I could do that. Or help design a PHP or static website to show your groups work. Or translate SAS code to python. Or help optimizing large queries for data analytics workflows.

Most of my work is small fixed contracts, ranging from $3k to $80k in my career.

LLC Website: https://crimede-coder.com/

Personal Portfolio: https://andrewpwheeler.com/c-v/datascience-portfolio/

Contact: andrew.wheeler@crimede-coder.com

apwheele··on Reducing late-night alcohol sales curbed all violent crimes by 23% in Baltimore
Reading the paper, this is pretty weak sauce findings. Here is the main scatterplot showing the fit lines, https://lh3.googleusercontent.com/pw/AP1GczOuepTKEj908Wc8Gkx...

The main findings are suggesting a slope change, which when you incorporate standard errors are just below 1 (the yellow line in the above graph). The model to me though is very noisy (should clearly have seasonal monthly terms, this should make estimates somewhat more precise), and is suggestive that the matched areas are not quite the same (they are substantively lower, although to be fair do appear to have similar seasonal patterns).

Biased because I have written a bit on this (IMO the pub health literature is biased to only conclude alcohol is bad, a similar paper saying "no effect" would be rejected at JAMA or other health journals). It is an interesting idea, I think it could probably be made stronger though (control for the same area during daytime, or a better synthetic control area). As is the 23% reduction is a bit wishful thinking I believe.

apwheele··on Ask HN: Scam IT Consulting Firms?
I get that part -- I don't get the bad website though. Like why have a http website? At least have a legit website and advertise your services no?

How did they get the contracts for outsourced work to begin with? (Is there some type of shell game that prompts a larger company to make these smaller companies?)

apwheele··on Ask HN: How to secure website for public launch
I have a similar background, and I just use a $5 a month Hostinger plan that manages the PHP server and I am quite happy with it. So it is just keeping my server side secrets in PHP in a way that makes sense.

Now, this does not allow me to say do python web-apps (that are not WASM). Hostinger has VPS for quite cheap I would consider if I needed that (if AWS lambda does not make sense, I did a python google cloud app engine for a month, https://crimede-coder.com/graphs/Dallas_Dashboard, and that was pricey, like $80 a month, whereas the WASM app is no additional cost). And I am sure there are other vendors that are similar (I am just happy with Hostinger).

So in terms of DDOS protection this is not so great, but that would not be a big deal to me. So site goes down, but I do not rack up a bill or anything.

For a google maps application, I not un-commonly see people put API keys in javascript client side (not good!) I mean it depends on what exactly you are doing, but if it is a public service that users do not sign into, just rate limiting the number of API queries in some PHP + database logic server side should be not too much work and reasonable to not rack up a surprise bill (I forget if google allows you to limit the API keys directly or if they will just rack up bills).

apwheele··on CDC sharply overestimates maternal death rate, new study finds
TLDR, maternal morbidity in US sources often widens the net to include things like drug overdoses. Whereas other sources do not, and that can explain the discrepancy between US and other nation maternal mortality rates.

Read this article the other day and has some breakdowns to show how many of those maternal deaths are for things not directly related to pregnancy issues (like embolism or eclampsia).

https://reviewtoaction.org/sites/default/files/2022-08/2014-...

apwheele··on Spc-kit: A toolkit for statistical process control using SQL
Agreed a bit of different use cases. (Or maybe should say the SPC approach is a better/more principled approach, whereas this is an easy graphical approach and I don't need to worry about "resetting" the CUSUM.)

I use the binomial CIs in dashboarding scenarios, so if the user selects a different subset of data, all the graphs are auto-updated (on the SQL side). Which when you get people subset into tinier slices of data it becomes more important.

So if you can upfront identify the processes you want to monitor, then when flagged have this send an alert. That is better than a dashboard forcing people to click on stuff and hope they identify a anomaly in the process.

← PreviousPage 5 of 8Next →