1,861 karma · joined February 1, 2013
Totally agree that sales teams can overreach though. Perhaps there should be a little “why are you signing up? Personal/work/etc” quiz when you sign up, then emails reduced accordingly.
> "by day of week"
This means you have a time series with daily resolution (one observation per day) and you expect Weekly Seasonality to matter. Model this as 7-period lag in your daily series.
> based on % of increase from other markets if no previous years
There are multiple markets, each with their own time series? Congratulations, you have Panel Data [0]. Do some regions have similar trends? Need to account for that correlation, maybe try a Mixed Model [1]
> percentage of growth from last 2 years
So there's a trend component (constant growth over time). Easy enough to fit the I term of an ARIMA model for this. You'll need to do some custom work to integrate this with your cross-regional correlations though.
> Average processing time
You'll want to model this at least as well as you're modelling the demand. That means seasonal effects, correlations between factories / warehouses, etc. PS any time you're dealing with a two-day weekend you'd better use at least 3 years of data in case major holidays happened to fall on a Saturday and Sunday, blindsiding you when it shows up on Monday this year.
> show the holidays
You'll definitely need to put together a calendar of major holidays for each region. These models will calculate the effect for each holiday. Specifically, the effect of the holiday AFTER accounting for the day of week, overall growth, time of year (season), and region. You might even get nifty charts like [2]
===============
Anyway that's the basics. You can take graduate math courses in just this kind of modelling. Easier - you can contract a decent statistician for a couple weeks to build the model for you. They'll be delighted that you can produce SQL queries with the relevant data, and their models will help you get a lot more value out of those queries.
===============
Resources 4 U
Generic time series reference:
[0] https://en.wikipedia.org/wiki/Panel_data
[1] https://en.wikipedia.org/wiki/Mixed_model
[2] https://assets.digitalocean.com/articles/eng_python/prophet/...
[3] https://en.wikipedia.org/wiki/Autoregressive_integrated_movi...
R tutorials:
[4] https://www.datascience.com/blog/introduction-to-forecasting...
[5] https://cran.r-project.org/web/views/TimeSeries.html (Particularly the "Forecasting and Univariate Modeling" section for your problem)
Python tutorials:
[6] https://machinelearningmastery.com/arima-for-time-series-for...
[7] http://www.seanabu.com/2016/03/22/time-series-seasonal-ARIMA...
[8] https://www.digitalocean.com/community/tutorials/a-guide-to-...
In your example, you should absolutely start by cleaning your data up and run some basic SQL aggregations and plotting volume over time. So you look that that and notice (1) volume is increasing over time, (2) some holidays bump a few days ahead but drive very low volume day of (3) weekends are higher, but the effect isn’t pronounced the whole year and (4) summer is better for you than winter except for the Christmas season. Now: it’s two days before Halloween, what’s our anticipated sales volume?
If you baked all those observations into an ARIMA model, it’s trivial to crank out a forecast with quantifiable accuracy. If you just have lines on a graph, it’s hard to pin down all the independent effects and recombine them for arbitrary scenarios.
The discrepancies appear when you start dealing with probabilistic stuff. Suddenly everything’s conditioned by everything else, and your complexity skyrockets. Example: it’s easy to find people with about average height and weight, but the more physical characteristics you measure (increasing dimensionality) the more like you are to find something statistically unusual, eg very long fingers. Very interesting Air Force research on this [0]
WRT to optimization, higher dimensions can help avoid the local minima problem. When you have 1000s if possible directions to go, it’s unlikely that none of them will yield a better answer.
[0] https://www.thestar.com/news/insight/2016/01/16/when-us-air-...
At that point it’s just experience to learn the rules of thumb that guide practical implementations.
https://github.com/caskroom/homebrew-cask/blob/dc825c8fe0cd3...
Oh dear, we’ve reinvented Facebook!
Seriously though, I think this is more about community than technology, and more about channel saturation than centralization.
I think the best shot for what you want is the re-distributed web and niche social networks (subreddits?). The peer-based web stuff will give you that feeling of being ahead of the masses on tech, and subreddits for your favorite hobby will let you connect with people who care about your random thing.
It’s a challenging strategic issue.
There are a couple of different approaches. One is DNS-over-TLS. That takes the existing DNS protocol and adds transport layer encryption. Another is DNS-over-HTTPS. It includes security but also all the modern enhancements like supporting other transport layers (e.g., QUIC) and new technologies like server HTTP/2 Server Push. Both DNS-over-TLS and DNS-over-HTTPS are open standards. And, at launch, we've ensured 1.1.1.1 supports both.
We think DNS-over-HTTPS is particularly promising — fast, easier to parse, and encrypted.
I was convinced it was hopelessly corrupted by the archiving process, but some of the ASCII art still works (mostly).
Link?
> the major concern about [Johns] is not so much that they pay for sex but that they seem blind to the plight of these women. They convince themselves that prostitution is a choice and that none of the women they see are exploited.
There’s a lot of exploitation in our society, and I don’t like any of it. Simple example: when I go to a restaurant, it’s very possible that the person serving my food works 2-3 jobs and still can’t afford health care. I hate that. I would like to be confident that everyone I meet was able to get basic necessities like healthcare.
Legalizing prostitution would open the door to reducing exploitation. Does the prostitute have a state issued sex worker ID? Are you paying at least the state-mandated minimum? Did you pay via a certified escrow service that has strict requirements to watch for common signs of abuse? Then no worries, you’re probably not supporting exploitation.
Compare that to what we have now, which is a total lack of transparency. Demand for sex is not going away. We need to prevent it from causing exploitation by creating a safe, legal option.
> but someone will still want to fuck 14 year olds
This is still wrong, we can still criminalize it. That’s fine. We can still give a lot of other sex workers way more safety and legitimacy than they have now. Don’t let the perfect be the enemy of the good.
There could be secondary effects that help the 14 year olds. (1) Most Johns would prefer the legal adult market, so that’s where the money will be. (2) There would be a bigger stigma to the illegal stuff because there are good legal alternatives, creating s higher barrier to entry. (3) institutions would develop to support sec workers, and they’d have resources and visibility to help the minors, more than exists now.
The point is we can decriminalize/ legalize a LOT more than we do now, and have a net positive impact.
Whatever policy we choose, it should have a clear goal and be evaluated against how well t achieves that goal. “Fewer deaths by tainted drugs” and “fewer sex workers living with abuse” is a good place to start. I think decriminalization work further both of those goals.
I think almost every vice would be less damaging to society if it was in the open. Polite society doesn't want to see sex work or drugs, but they still exist. Hiding them makes things much worse for the people directly involved. It's trafficked kids with broken immigration status who are more scared of the cops than their captors. It's drug addicts who OD on tainted drugs.
Bring it all into the open. Have the government certify providers directly. Crack down on unauthorized middlemen. Tax it. Use the taxes to pay for programs that help people leave when they realize they can have a better life without it.
We need the classic American Market here: free trade enabled by regulation that ensures market quality and protects participants from fraud.
Unfortunately I don't think this is politically possible. It would take a long, well funded campaign. The people who are willing to do that kind of work are motivated by stories of individual tragedy and focus on draconian solutions like this mess of a law. The people who would push for openness can make more money elsewhere, and don't want to make their name "Advocating for drugs & sex."
Frustrating.
At minimum the Cambridge Analytica scandal will have to be addressed. If users are leaving the platform, that would need to be reported too.
Facebook has been aggressively monetizing user data for years. They are just one player in an entire industry built around this business model.
The existence of this industry is most obvious to technically literate, who you can generally identify by their use of strong ad blockers and password managers. But it’s been reported on before [0]. Online privacy is not a new concern... just look at the “Facebook is listening to me” meme.
So what I want to know is: why now? Why Cambridge Analytica? Why Facebook?
Here’s my best take so far.
1) Facebook’s user base is so big that it’s a relevant political constituency, and thus democratic governments have a reason to care.
2) Facebook creates an expectation of privacy and illusion of control that doesn’t exist with public-first platforms like Twitter
3) Cambridge Analytica is a scummy company in many respects, not just their work with Facebook data. They got lots of Facebook data from a third party who probably wasn’t authorized to sell it to them. This makes them a good candidate for regulators to make an example of.
4) Cambridge Analytica is closely tied to the Trump and Brexit campaigns, both of which are regarded as “dangerous perversions of democracy using lies to exploit vulnerable people” by exactly the kinds of political and media organizations that are driving this story.
Overall, I think this is a “Pigs get fat, Hogs get slaughtered” situation. The industry’s toxic practices are finally causing enough damage that institutions responsible for protecting the public (government, real media) are responding.
Bravo, I say.
[0] WaPo has an article explaining this on the front page today https://www.washingtonpost.com/business/economy/facebooks-ru...
- government managers are evaluated on budget and headcount because the core services they provide are hard to measure. Thus they don’t want new software to help them be more efficient, unless the alternative is catastrophic, visible failure to provide their service.
- Lawyers charge by the hour. They don’t want new software to dramatically reduce the time spent in discovery. Then they would have to bill for outcomes (cases won) and there are too many things out of their control.
Basically nobody wants to charge for value, because value is hard to quantify accurately and quickly. You wind up showing costs to prove value, so nobody wants to reduce costs.
When I take a MOOC, I'm pledging at least 5 hours a week. That covers 2 hours of videos, 2 hours of homework and 1 hour of additional research / review.
If I pledge 5 hours /week to a traditional class, I don't get nearly as much learning. I have to commute to the class, which is at least 30 minutes each way. Now my 5 hours buys me 2 hours of class, 2 hours of commuting and 1 hour of homework. I've lost an hour of homework and an hour of research.
I know which one is better for me.
Go to school when you feel you need something they can teach you, and not before.
I think this is about the point where my wife would have killed the project.
Here are some reasons why:
1) loss of corporate identity / camaraderie
2) reduced working training & quality... and loss of control over how work it performed. If you retain control over work methods, they’re supposed to be employees
3) double downward pressure on wages as companies compete and then the contracting firm takes a cut. Pressure increases as subcontractors show up
4) subcontracting! You hire firms with no capabilities, which then hire subcontractors who hire people who know what they’re doing (maybe, see above). Why are these companies making profits? It should all go to the workers.
5) false flexibility. Your HR system might be painful, but good luck getting rid of a toxic contractor if they’re the cheapest person who can do the work. You can take your contract to a new company but they’ll hire the same people you already had.
Contracting obviously has its place - no firm can do everything- but staff replacement just sucks.
-
One nuance - these p2p protocols are not as simple as HTTP, FTP, etc. They require persistent connections to many peers and significant local storage. This eats up more local resources, to the point where making in default-on might cause significant performance hit.
I'm not sure how IPFS addresses this, but one method is to have an oracle that reports DMCA takedowns. Nodes can achieve takedown compliance by subscribing to the blacklist.
Obviously there are lots of technical challenges. Just a few:
- Who gets to add entries to the blacklist? - How do infringement victims know that's the right person to contact? - How do you avoid creating a central point of failure? (at minimum nodes need ability to choose their blacklist provider) - Since it's all content-addressed, how do you handle near duplicates? e.g. same movie encoded in different formats (basically have to punt on this one unless you implement some kind of contentID, which is a whole other problem)
Anyway, you need to have some figleaf policy so people have a clear way to opt-in to compliance.
I would be completely skeptical of this approach except that it appears to work for adblockers. Extensions like uBlock Origin let users choose from a several third-party blacklists, with sensible defaults. Of course, there are fewer ad servers than pirate movie copies...