The B-schools are basically forced by some standards efforts to teach some courses that are essentially applied math.
One might guess that the B-schools want to be professional or clinical but, instead, they tend to want to see themselves as applied social science, and social science has physics envy and wants to be mathematical. Their favorite math is linear statistics. So typically there's a lot of such statistics in the research at B-schools. And typically the B-school profs are not very good at math or statistics.
B-schools also want to dabble in computing -- if you can teach some courses in computing, even as electives, then you might get liked for that.
No doubt some B-schools would like someone who could teach courses in financial engineering, say, with Brownian motion, stochastic integration, Black-Scholes and generalizations -- basically the Brownian motion solution to the Dirichlet problem. So, if you like mathematical finance, then maybe teach it in a B-school.
DC may still be a good area for applied math: So, submit a Civil Service application and also send copies to the usual suspects -- NIST, various DoE labs, various DoD labs, various groups interested in economics or mathematical finance, that is, maybe the Federal Reserve (also try the regional Fed banks), the Social Security Administration, etc. If you want, try NSA, CIA, DIA, FBI, etc. -- no telling what they might want to do with some applied math. Also try the JHU/APL.
There have long been lots of Beltway Bandit shops because commonly Congress gives the departments of the Federal Government more money to spend than full time head count to spend it on so that a lot of the money and/or work goes through companies. E.g., Edward Snowden worked for Booz-Allen or some such.
For technical jobs around DC, it used to be that the job ads in The Washington Post were good places to look.
There is a chance that Google, Facebook, Yahoo and some others are interested in some projects that might be able to use some high end applied math. A problem may be that the other workers there are not very good at math and, really, make a mess out of the work, the opportunities, the positions, etc., and that can be bad for everyone.
It's not clear that everything in mathematical finance is a rat race. I doubt that James Simons ran a rat race shop.
Houston has some important slots for optimization, e.g., as in the work of Princeton Chem Eng prof Floudas. E.g., here's all the crude oil inputs we have available and the costs, and here's all the prices for all the possible refinery products. So, say what crude oil to buy and what products to sell to maximize earnings. That's an old problem, but likely people are still working on it.
The airlines have some very serious problems in fleet and crew scheduling, and first cut it's integer linear programming. But with uncertainties, if you want to handle them at all realistically, the formulations and solutions can become challenging in all respects. A better solution can show its value just on the computer and get taken seriously. For some airline spending $100 million a month just on jet fuel, saving a few percent can pay for some nice computing, work, etc.
Maybe look at what the operations research people are doing these days in scheduling for trucks, airplanes, cargo ships, etc.
No doubt Amazon, Wal-Mart, etc. have some good problems that are generalization of the old transportation problem -- the one Kantorovich got a Nobel prize for. Now we regard that problem as least cost flows on a network and use the simplex algorithm to solve it and where a basic feasible solution corresponds to a spanning tree of arcs on the network.
My view is that broadly now being handy with probability and probabilistic modeling of real situations is valuable, not that you can expect anyone else to know this so that, really, all you can sell are results, not the work.
E.g., with the Kolmogorov definition of a random variable, a function from a measurable space to the reals measurable with respect to the Borel subsets of the reals and a sigma algebra on the measurable space, go get a number, any number, and then regard it as the value of a random variable, that is, intuitively, one number among others that might have been observed. Now you know the first step in analyzing any given data -- call each number the value of a random variable. Then look for some assumptions, independence, orthogonality, or other relationships among the random variables. Keep the weak law of large numbers at your side -- that is, get some independent, identically distributed samples and take an average. There's more! Uh, the set of all real valued random variables X so that E[X^2] is finite forms a Hilbert space. The inner product is (X,Y) = E[XY]. So, yup, that completeness holds is a bit amazing, but it does and the proof is mostly just the Minkowski inequality from the inner product. Then in a Hilbert space we can do projections, and they are approximations, e.g., least squares.
Currently the Internet ad people and their ad targeting should be an opportunity to do some valuable applied math.
For what to study beyond the usual ugrad and Master's pure/applied math, I'd say the math sciences, the math of operations research or systems analysis or electronic engineering. So, linear algebra, numerical linear algebra, linear systems of various kinds, optimization (linear, integer linear, non-linear, network linear programming), etc.
My view is that soon computing will run aground due to running out of any additional utility from just their usual intuitive and heuristic approaches to solving practical problems and need math. But don't hold you breath.
Maybe you can have a good, long term career, say, long enough to have kids and get them through college, at some one organization. Maybe. Maybe for some DoD lab. Maybe.
Otherwise I have to suspect that in the end you will have to be a businessman where you think of the product/service, develop it, deliver it to your customers, and get the revenue. Then you get to use whatever math helps without trying to talk some non-mathematical manager into letting you try, and then you just deliver the results, not the math itself, to your customers.
In the short term, I'd suggest get a job for a salary, keep one foot in pure/applied math, keep the other foot in computing, try to make a career out of the job, but really expect to have to do, and, thus, to look for a way to do, a startup based on applied math and computing. There you just use the math as an advantage, maybe the crucial core -- still the rest that is more routine is also important.
For how "lucrative", essentially that's up to you! Broadly, can't really expect someone else to create a good job in applied math for you. Instead, you have to create the job you want and then take that job, do well with it, and build a career. You can do this starting a pizza shop; with math and computing, they should be advantages. E.g., with some abstract algebra applied to coding theory, A. Viterbi built Qualcomm.
If you know someone who knows someone, then maybe could get into some niche slot somewhere, maybe on the staff of a committee of Congress.
FedEx had candidate investor General Dynamics (GD), and GD had sent two of their guys, one aero engineer and one finance guy, to help FedEx. FedEx needed the GD investment.
At one point, the Board wanted some revenue projections. I wasn't asked to get involved and didn't want to, but no one had any ideas of how to do the projections except just draw a free hand line on a graph based on hopes, dreams, intentions, guesses, etc.
So, I thought: We know (1) the current revenue. We know (2) the revenue when the planned service to 90 US cities with 33 airplanes is full. So, the projections are roughly how interpolate between (1) and (2).
For how the interpolation will go, we have (3) current customers and (4) the rest of the customers we will have when we have (2) and are full.
Then say that the growth is from current customers (3) talking to the rest of the customers (4). That is, the rate of growth is directly proportional to the number of (happy) customers (3) talking to the rest of the customers we will have (4).
So, let t denote time in days with t = 0 corresponding to the present. Let the revenue at time t be y(t). Let the revenue from (2), being full, be b dollars a day. Then at time t, the growth rate is y'(t), that is, d/dt y(t). So, we have that
y'(t) = k y(t) ( b - y(t) )
How 'bout that! Maybe it's so simple it's almost just a joke, but the rest of the ideas floating around the office are much worse!
So, sure, it's an initial value problem for a first order, linear ordinary differential equation but, really, is so simple can just use freshman calculus directly and find the closed form solution, right, some exponentials I omit here.
So, on a Friday the Senior Vice President (SVP) for Planning and I got out some graph paper, picked a value for k, and drew the graph.
The next day, Saturday, I was in my office, and at noon I got a phone call asking if I knew about the revenue projections. Well, the SVP was traveling, and, yes, I knew. So, I was asked if I could come over to the HQ offices and explain. So, I got in my Camaro hot rod and drove over.
When I got there, people were unhappy. The two GD guys were standing in the hall with their bags packed.
I was led to the graph, given a time, and asked to reproduce the value on the graph at that time. So, I had brought my HP scientific calculator, punched the buttons, and got the value on the graph. I did that a few more times, and people started to be happy again.
It turned out that the Board meeting had been that Saturday morning; the graph had been presented; and the two GD guys had asked how it had been calculated.
Well, all the FedEx people there tried to figure out how the graph had been calculated and went on for a few hours without success, and then GD guys lost patience, got plane reservations back to Texas, went to their rented rooms and packed their bags, and as a last chance returned to the HQ offices to see if FedEx had an explanation for the graph. FedEx was about to die.
That's when I arrived and explained the graph.
The FedEx guys stayed, and FedEx was saved.
I never got thanked! Really my impression was that the rest of the top of FedEx had concluded that I'd been dangerous, maybe had 'hooked' the company.
And I'd never been invited to the Board meeting. So, I began to conclude that my contribution was so resented that the top of FedEx would rather see all of FedEx go under than let me help.
The promised FedEx stock was already 18 months late. I was in Memphis while my wife was in her Ph.D. program at Johns Hopkins near our home in Maryland. So, to heck with FedEx -- I went home and to Hopkins for my Ph.D.
So, that's an example of applied math in US business! E.g., I was the only one around who still knew freshman calculus! And, heck, I'd never even taken freshman calculus, had studied it on my own and started with sophomore calculus!
A broad lesson is, for anything very technical, have a tough time just making a contribution in the middle of a lot of operational and management people who forgot freshman calculus -- such people will feel out of control and resentful.
So, instead, for technical contributions, need an appropriate organizational structure, maybe be in a research division and out of the main line operations or management, maybe be a respected external vendor, maybe be in a profession complete with legal liability and licensing, etc. And charge a LOT.
Of course, broadly, the flip side of such technical incompetence, at the level of just freshman calculus, should be an opportunity!
There were several other cases where I did some good technical work on an important problem but resentment by others and competitive office politics kept me from getting credit or actually blocked my contribution. In the field of organizational behavior, such politics is called goal subordination, that is, someone finds it in their interest in the company to hurt the company, subordinate the good of the company to their own good.
Broadly, being a really good technical contributor among a lot of less technical people is a great way to create enemies who will sabotage your career, not friends who will help it.
So, net, usually have to make a technical contribution from outside the pecking order of the organization. That is, be a vendor.
Still better, just sell a product or service the customers will like but where the customers need not be aware of anything technical in the work.
That story about FedEx is hilarious!
> Having started working mathematically recently it seems to me that even though I can propose a solution which isn't the best, they are usually better than what is currently available. Is that a common experience as a working mathematician.
Sure. For the politics, just report the savings. The stuff about the optimal solution, that is, its, say, moral or ethical value, is lost on business people. With some good common sense, the business people just want to know how much has been saved (or extra earned). If you can also have, say, a lower bound on the possible savings and get close to that, then you have in practice a stopping criterion. Or, what in practice is really difficult about the NP-hard problems is usually saving the very last possible tiny fraction of one penny and proving that did that. Instead, just concentrate on saving, say, the first $5 million a month on, say, the $100 million a month of jet fuel for an airline.
For a while in optimization, having the optimal solution was made a moral objective with anything that wasted even 0.0001 pennies regarded as a sin. Religious nonsense.
Really sloppy work is not good, but close approximation usually is plenty good.
For not the best, that's the other time I kept FedEx from going out of business: The founder, Fred W. Smith, spent the afternoon in his office trying to schedule the fleet, staggered out exhausted, said "We need a computer", and a college friend of mine heard that plea and called me.
Sure, digging in, I heard a lot about optimal solutions. But I knew that first just get something that helps, say, so can type in a schedule, have the software check it out for, say, planes being over loaded or having to fly too far, get the arrival times, etc., and print out all the details. So, I typed in 6000 lines of code to do that. Right, great circle distances are from just the law of cosines for spherical triangles!
Soon the Board wanted a schedule; biggie Board members doubted that the scheduling could even be done; biggie funding was being held up; the company was at risk.
So, I got a call from the same guy who called me to the Board meeting for the revenue projections, Roger Frock (he wrote a book about FedEx), and he and one short, easy evening he and I used my software to develop a schedule for the whole planned system of 90 cities and 33 airplanes. It printed out and looked nice.
At the next senior staff meeting, Fred's reaction to the printout was "Amazing document -- solved the most important problem facing Federal Express". Our same two guys from General Dynamics went over the schedule carefully and announced "It's a little tight in a few places but it's flyable". The Board was pleased; the funding was enabled. That was the first time I saved FedEx.
So, I got into how to do better scheduling. So, for one period from, say, 5 PM to midnight (again from 1 AM to, say, 8 AM) set up a table with one row for each city to be served, 90 cities, and one column for each candidate airplane tour from Memphis and back. So, may have some thousands of columns. Considering all fine details want to account for, and take expectations for random costs, find for each of the candidate tours the cost (first cut, assume that if a plane goes to a city, then it completely serves that city).
Then for the schedule, want to pick no more than 33 columns from the 1000s so that all 90 cities are served and total cost is minimized.
So, each column covers some cities, and want to pick columns that cover all 90 cities -- so have a set covering problem. Right, it's in NP-complete.
Well, that table is essentially the matrix in a 0-1 integer linear programming least cost optimization problem. There have one unknown variable for each of the columns, and the variable is 1 if use that column and 0 otherwise. If relax the 0-1 constraint, then have a fairly routine linear programming problem but will likely end up with some fractional airplanes allocated. Could use branch and bound to get to a good or optimal 0-1 solution.
Should have saved FedEx a huge bundle. Now, with software such as the IBM Optimization Subroutine Library (OSL), C-PLEX (now owned by IBM but from Robert Bixby at Rice in Dallas), etc., that would be a fun and likely fairly easy problem.
Alas, for proposing that work as my project, I got too much guff and flack back from too much of the top of FedEx, wasn't getting much thanks for the two times I'd saved the company, was not seeing the promised stock, was away from my home and wife, was in a company close to going belly up, saw a lot of dumb work, so went for my Ph.D.
For what companies to go to to get going on a career in such applied math, I can't think of any. Mostly people and their organizations would rather waste money than get involved with mathematicians doing work only mathematicians understand.
Again, if want to make math pay, just sell the results, usually as a vendor outside the customer.
If are inside the company, then likely no good work will go unpunished. Basically to be successful, you will need the strong personal support of the CEO and hopefully the Board -- literally. And the CEO will routinely have to go around and break 2 x 4s over the heads of recalcitrant middle managers to see if they are just asleep or really dead -- otherwise those middle managers can sabotage you with high determination and creativity.
E.g., twice I saved FedEx from going out of business and had much more that would have saved FedEx maybe tens of millions of dollars a year, but I got flack and push back and contempt and none of the promised stock.
Such are some of the challenges in getting stuff done. But successful people have been overcoming challenges, some much worse than I faced at FedEx, for many centuries.
Now that you know about the possibilities of some of the absurd challenges, don't be surprised. Really, one of the best rules is just to keep your good technical work essentially a secret until you have a solution that is just too darned good to refuse, and then pull back the curtain, let people see the results and also see that it's too late to fight with you.
Better still, just provide such results as a business and own the business. E.g., in some cases, just do the work for free, show the results, and then announce "You can have this for your real operations for a fee of only 15% of the savings."
Once there was an in-house research group that was successful. It was for WestVaco paper -- think specialty papers, e.g., coated papers for milk cartons. So, they had Ph.D. chem eng guys running their plants in jungles, etc., that is, where the trees were, but it was a family owned business with HQ in NYC.
The research shop was in Maryland, and I got an explanation from them: Each year Research went to NYC for their budget. Always the CEO, family guy, offered much more money for the Research shop than the shop was willing to accept (first-cut, good budget situation!).
Why? The rules: Research picks their own projects. Projects tend to run from a few months to a few years. One project in 10 gets to operations. When Research has a project ready for operations, they approach the relevant operational managers. In the first three years, savings are allocated half to Research and half to the operational unit. After three years, all the savings are allocated to the operational unit. With this accounting, Research was returning $3 to the company for each $1 in their budget. So, sure, HQ wanted to increase the budget, but Research didn't want to have to find ways to return $3 for each extra budget $1 so didn't want a bigger budget!
It is just crucial that the CEO is supporting such in-house innovation. The rules are also important.
Ah, while I was in grad school, to make enough money to support my wife and I while we both finished our Ph.D.s, I took a job at a DC area think tank working for the US Navy.
At one point our problem sponsor group wanted an analysis of a special, worrisome scenario of global nuclear war but limited to sea -- they wanted to know how long the US SSBN fleet could survive if it was just held in reserve and didn't shoot its missiles with nuke warheads. And they wanted the results in two weeks.
Gee, two weeks to model global nuclear war limited to sea! How generous! Good that they were not in a real hurry! Besides it was good timing since the day after the due date my wife had already gotten us reservations for a vacation at Shenandoah!
Well, there was a guy B. Koopman who in WWII had written a report OEG-56 on finding things at sea. So, given A and B wandering around at sea, when might they have an encounter? Well, Koopman took the area of the sea, the two velocities, and the detection radius and came up with an arrival rate for a Poisson process. Hmm ....
So, list the Red forces and Blue forces (the US SSBNs were part of the Blue forces), that is, list airplanes, helicopters, destroyers, battleships, attack submarines, etc., the speed of each, and the detection radius. Then have a table with a row for each Blue weapon type and a column for each Red weapon type and the detection radius and, if there was an encounter, the probability that one, the other, or both died.
Now tap lightly with an independence assumption (maybe not wildly unjustified), realize that a sum of independent Poisson processes is again a Poisson process with arrival rate the sum of the contributing arrival rates, and get for the state of the war a continuous time, discrete state space (very large from a combinatorial explosion) Markov process subordinated to a Poisson process. So, yes, there is a closed form solution as a matrix exponential for the transition probabilities from state to state, but that matrix exponential is absurdly large. Instead using Monte Carlo to run off, say, 500 sample paths was easy. I typed in the code.
Prof J. Keilson did a technical review. He said, "There's no way you can fathom that enormous state space.". I said, "At each point in time, say, 10 days into the war, the number of SSBNs remaining is a random variable. Moreover it is bounded so has finite expectation and finite variance. So, the law of large numbers applies. So, run off 500 independent, identically distributed samples, add them up, divide by 500, and get the expectation within a gnat's ass nearly all the time. Really the Monte Carlo puts the effort where the action is.". He agreed with me and passed my work.
The Navy got their results on time, and my wife got her vacation on time.
Later I was told that my work had been sold to a leading US intelligence agency. I could tell you which one, but then I'd have to ...!
I thought that the whole effort, which ignored so much maybe crucially important detail, was close to a joke, but, surprisingly, the output did look reasonable. Also, the two weeks was also so short it made the whole exercise next to absurd, another joke.
The group I was working for didn't trust me to get results on time so had another person working independently. Well, at the end of the two weeks, the other person had nothing at all. So, I'd won.
Office politics: One guy said "You write nice computer programs" and otherwise I got no thanks but just some jealousy, contempt, and silence. So, soon afterward when I'd gotten my Ph.D., I wanted to leave and did.
Again, no good deed need go unpunished.
Again, own the business, use math as an internal secret sauce and technological advantage, and sell just the results.
Again, thanks so much. One last question, what are you doing with your life right now? It seems like you have had a pretty broad career and I am interested in the end game. If you are retired, what is your lifestyle like and in particular how has your career impacted your life as a retiree.
For now, sure, I know some math and can stir up some more, and I know some computing and can learn more.
So, I've got a 1.8 GHz AMD single core processor, Windows XP Professional SP3 (I have an official copy of Windows 7 Professional on DVD but have seen no great reason to go to all the trouble to install it and rebuild all my software environment yet), three hard disks, 100 million files, .NET Framework 4.0, a good text editor (KEdit), Visual Basic .NET (comes with the .NET Framework), ASP.NET, ADO.NET, IIS (low level Web server), SQL Server Express, etc.
I thought, why not go after about 2/3rds of Internet search, the safe for work part served at best poorly by looking for keywords/phrases?
So, how to do that? Not with just routine software! And not with anything commonly talked about for search!
I stirred up some math, typed the theorems and proofs into Knuth's TeX, worked up a scalable architecture, and typed in the software.
Maybe I should have used Redis, but instead I thought that writing my own session state server would be faster than even understanding Redis. So, my session state server is single threaded ("Look, Ma, no concurrency problems"), for faster lookup rates is trivial to run as several instances as in sharding, and is just some TCP/IP sockets, some de/serialization of instances of my session state class, and uses two instances of a .NET collection class. Simple.
It's fast! A server for less than $1500 should be able to keep session state for an hour of inactivity for each user and do the session state work for sending 5000+ Web pages a second. Two standard racks should be able to handle session state for the world.
The rest of the software is also readily scalable also from just simple sharding.
Currently I have one bug in one Web page -- it's not handling session state just right! But I have the fix in code in another Web page and should copy it over today!
My interactive development environment is just KEdit with about 100 macros and some careful use of file system directories.
I'm using SQL Server only to record the data from the users and for the results of some batch computations; at one point I actually do make use of a transaction; the data for the searches is drawn from SQL Server with a batch program (run it maybe once a day); some solid state drives (write rarely, read thousands of times a second) should do wonders for the data for the searches.
I was about to fix the bug in the Web page but took some opportunities to gather some good initial data. The site will start focused and only slowly grow to be comprehensive -- right, at first do some things that don't scale and please some niche group of users a lot instead of trying to please 2+ billion users a little.
Currently the database has only some meaningless data I put in for first testing of the software. It's about time to load in some of the good initial data I have. Then give a critical review, go live, etc.
It's getting there.
All the work uniquely mine has been fast, fun, and easy, but the whole project has taken far, far, far too long. Why? I worked through about a cubic foot of books and 6000+ Web pages of documentation of Windows, .NET, Visual Basic .NET, ASP.NET, ADO.NET, SQL Server, etc.
The main problems: (1) Badly written, obscure documentation (worst bottleneck in the future of computing); (2) computer viruses; (3) SQL Server installation bugs destroying my boot partition requiring rebuilding starting with the XP DVD (barbed wire enema with an unanesthetized upper molar root canal procedure); (4) SQL Server management and administration (e.g., a week of throwing stuff against a wall to see what sticks just to get a SQL Server connection string that will let code for a server side Web page connect with SQL Server); (5) clean, smooth means of system backup and recovery (including for both user data and bootable partitions); (6) Sony DVD drives that quit for no good reason (and inability to buy more IDE DVD drives).
Good stuff: (1) KEdit and its macro language; (2) the scripting language Rexx (Microsoft's PowerShell may also be terrific but have yet to move to it); (3) NTFS (fantastic); (4) Visual Basic .NET design, functionality, speed of compilation, compiler error messages, minimal bugs (sweetheart language); (5) what ASP.NET does when it compiles a Visual Basic program (enough for a really nice IDE); (6) NTBACKUP (once understand how to use it, e.g., do have to ask to save "system state", whatever the heck that is, or the saved copy, restored, won't boot -- learn this and how to get around it the hard way, weeks of work); (7) XCOPY; (8) the tools to have server side Web page code write to a log file; (9) Firefox (except for virus vulnerabilities); (10) the classes in the .NET Framework (once learn how to learn about them and use them); (11) Adobe Acrobat (except for virus vulnerabilities); (12) the ability of XP to find device drivers and recognize new devices; (13) Microsoft's anti-virus Safety Scanner (if only from CP67/CMS and Multics, there should be no virus vulnerabilities, but since there are the safety scanner is terrific to have); (14) Knuth's TeX; (15) the Western Digital Passport Ultra 2 TB USB drive!
I'm not retired or retiring! Likely I'm still 100% unemployable at anything that would pay enough to let me keep a car going to commute to the job. Y Combinator and VCs want nothing to do with me.
But if I can get my Web site up to a search a second, on a server for about $2000, I will be in decent shape financially and on the way to organic growth for my business and much more.
Then I'll get a nice house, a building for some cars, at least one Corvette, visit the rest of the family still alive, take off two weeks to pig out on lobster in Maine, get some good grape juice from between Beaune and Dijon, do some cooking, give some dinner parties, go to concerts and operas, continue with the business, go to seminars on mathematical physics, etc.
I'll implement and deploy my server farm monitoring techniques and maybe spin it off as a separate business.
I have some guesses for some approaches to real AI and might try to implement those.
And I will get a kitty cat! We'll see!
Good luck on your work with math. Maybe what I typed in here will help; I wish I'd known all that when I started.
Hi graycat,
I'm an applied math & CS undergrad at Caltech and I love all of your posts, especially the ones about how data center scale computing really needs to embrace statistics. The FedEx stories are also excellent. I'm personally interested in high performance computing and machine learning, but I'm also interested in solving "real problems" and like how you seem to focus on the actual value of the applications. I love the feeling of engineering solutions to mathematical problems, and this seems to be something that you also enjoy.
I'd really like to hear more about your career, research, and also the startup that you're working on. I'd be very happy if you shoot me an email at eric@ericmart.in
Best, Eric
I really enjoy your posts and your writing style. Thanks much. I am currently working on some math problems as a part of a business that makes accessible to Indians who do not have access to structured banking services. Would love to discuss it with you, if you are inclined. My email is takenottie at google's mail. Thanks.
Well, in part we aimed too high: My wife was brilliant, quite broadly -- yes, Valedictorian, Summa Cum Laude, Woodrow Wilson, PBK, piano, clarinet, voice, prizes in cooking, sewing, raising chickens. But her family had her try to be perfect and to dedicate herself to saving the world.
She wanted a Ph.D. in mathematical sociology to do social engineering of social change to save the world and, thus, get her praise, acceptance, and emotional and financial security. And those concerns filled her plate.
Well, at the roots there some anxiety, from nature and/or nurture, was involved, then some perfectionism and some fear of criticism from powerful, influential people -- net, call it a special case of social phobia. That brought stress which can bring depression. That slows the work and in her case caused more stress and more depression then clinical depression. She was in a clinical depression the day she got her Ph.D. She never recovered and, net, didn't make it.
Took me a while to reinvent and learn the basics of clinical psychology to understand what was going on and what to do about it. I did learn but nearly always too late.
In trying to have a stable job so that I could take care of her, I took a job at IBM's Watson lab, in what we called artificial intelligence (AI) to do monitoring and management of large server farms and networks.
Then IBM got sick and the lab phone book went from 4500 full time names down to 1000 or so. The guy who hired me, a big star who was deliberately ignored by the higher ups, left for greener pastures -- eventually ended up with a nice place in Malibu.
IBM Watson Research was run by a clique of people who stuck together and blocked out everyone else. At one point it appeared that the company HQ in Armonk tried to correct that situation.
Instead of or better than the AI, I had some ideas: Detecting a problem in a server farm or network is necessarily essentially a statistical hypothesis test with Type I (false alarm) and Type II (missed detection) errors. So, want hypothesis tests that keep down the rates of the errors.
Since there's a lot of data readily available and want to exploit it to keep down the rates of the errors, also want multi-variate inputs -- nearly all hypothesis tests have only univariate inputs. Also for such multi-variate data can't hope to know any of the probability distributions so need tests that are distribution-free.
I don't think there were any such tests, so I dreamed up some. I used some group theory, summed used the classic S. Ulam, guy on the left in
http://www-history.mcs.st-and.ac.uk/BigPictures/Ulam_Feynman...
result LeCam called tightness (see P. Billingsley, Convergence of Probability Measures), and was able to permit selecting false alarm rate in advance and then getting that rate exactly. Asking for all of the Neyman-Pearson result would take more data than we had any chance of having, but there was a somewhat useful sense in which my work gave asymptotically, for any selected false alarm rate, the highest possible detection rate with that false alarm rate.
I cooked up some synthetic data that was challenging -- the critical region was something like the red squares on a checkerboard in several dimensions. My work did fine.
I got some data from a cluster of computers at Allstate, wrote some prototype software, and confirmed the false alarm rate empirically. And I cooked up an algorithm to make the computations nicely fast (in part I used k-D trees -- reinvented those -- a few years earlier and k-D trees would have been mine).
Politics: A guy in the clique up there didn't like me. But it took two levels of management to fire someone, so he reorganized to put me under a wuss who would go along with firing me. They claimed my research was not publishable (reviewed by a guy in the clique who admitted he couldn't read my math but claimed to have found someone who could but found nothing wrong with my paper) and walked me out the door.
The next day the wuss was demoted out of management. Two weeks later the main nasty guy was moved down to have him under an additional level of management and given a six month performance plan which he failed. He was demoted out of management -- lost his corner office, budget, secretary, and 55 subordinates.
I got a PC and Knuth's TeX and submitted a paper on my research. Since the paper had some measure theory in it, e.g., Ulam's result, much of the computer science community couldn't read it. But the journal that offered to review the paper kept at it; apparently the editor in chief walked the paper around his campus, to a CS department to see if the problem was important and to a math department to see if the math was correct, and accepted the paper. He invited me to present at a conference he was running, but I didn't want to bother going. The paper was published. IBM was wrong.
So it appears that I have the world's only collection, and it's large, of statistical hypothesis tests that are both multi-variate and distribution-free, with some nice properties, with a fast algorithm, with some confirmation of the false alarm rate calculations from some real data, etc.
Asymptotically the critical region can be a multi-dimensional fractal. Nice.
People should my work. I did give a talk on the work at the main NASDAQ site in Trumbull, CT.
IBM didn't pay very well, and cost of living was high -- I'd always saved money, even in grad school, but I lost quite a lot of money working at IBM.
When I joined IBM, it'd just won the Nobel prize in physics two years in a row and had a long string of being "the most admired company in the world". Now I'd advise anyone just to stay the heck away, a long way away.
But being pushed out of IBM and age left me 100% permanently unemployable. I sent 1000+ resumes. Zip, zilch, zero.
If in computing, be sure by age 40, hopefully by age 35, to have a rock solid stable career and/or be wealthy. So, really about have to own your own business and make it successful.
I work for an extremely large defense contractor, and it never ceases to amaze me how many of my fellow engineers have forgotten the mathematics they were taught in high school! I know many of them took AP courses and learned calculus there, but they have completely and totally forgotten it.
At one time I wrote Fisher Black at GS and got a nice letter back from him, I still have, saying that he saw no opportunities for applied math on Wall Street!
I didn't hear about James Simons until much later.