agentcoops

Really fantastic advice. The only corollary that I'll over-emphasize to those reading: do not rely on ChatGPT and learn to work with the APIs directly -- or tools built explicitly for your purpose.

I've also been working with the APIs directly for about that time and I only get more depressed daily reading the horrible takes about GPT's problems based only off of ChatGPT (and hearing about how even many companies try to use the product). GPT has forever changed how I interact with computers.

If you're already a programmer, there's no excuse for not working directly with the APIs. Learn about context and prosper.

Further evidence of my justified true belief that the world will rapidly divide between those who already had programmatic control over their computers, now infinitely more productive through direct access to the GPT APIs (to say nothing of the firms that will use LLMs only internally, trained on their own codebase for example) and those who believe GPT=ChatGPT, which will slowly become but another conduit for ads through plugins. It's quite sad when even Nature magazine becomes a spokesperson for the latter group, who will continue to believe that LLMs are a racket on the order of Bitcoin.

Many comments miss the mark as they fail to make the crucial distinction between ChatGPT and GPT-4. GPT-4 is the underlying model one can indeed have direct access to on a pay-per-request pricing scheme. ChatGPT is an application built on top of GPT-4 which manages how the 'context' of your interaction is passed in on a per-request basis. I don't doubt the spokesperson for a minute: from my own experience, the underlying GPT-4 models have not changed and I sincerely believe that OpenAI will be careful on this front, given that they are aiming to build a once-in-a-generation company that provides a stable platform for other firms to build products on top of.

The ChatGPT application, on the other hand, and how it manages context etc has certainly changed in the intervening time. That is completely expected as even and perhaps especially OpenAI is figuring out how to build applications on top of LLMs, which means balancing how one can get the best quality results out of the model while making ChatGPT in particular a profitable business.

Stratechery has analyzed this problem for OpenAI in the most detail I've seen. I imagine the company is in something of a bind figuring out how to invest between the APIs themselves and ChatGPT. On the one hand, the latter is incredibly successful as a consumer app with a lead it will be difficult for rivals to catchup with and it is likely plugins will provide a good revenue basis. On the other hand, there is certainly a greater business opportunity in being the foundation for an entire generation of AI products and taking BPs off of revenues -- if and only if GPT4 indeed has a significant moat over the opensource alternatives. For the moment, it would seem they will have to hedge both bets as we see how the consumer space and the competition between models heats-up.

We’ll see how the incentives play out vis-a-vis the tension between quality (especially for demanding non-consumer tasks), unrestricted API access (vs “chat” ux) and cost. I fear the Stratechery article might be right and that ChatGPT becoming a consumer success will be a curse in the end.

I’ve spent way too much time (and money) on the OpenAI API and spoken to enough non-technical people to realize now that ChatGPT has in some ways really mislead people about the technology. That is, while it’s impressive it can answer cold questions at all, the groundbreaking results are reasoning and transforming texts “in context”, which you don’t have control over easily with ChatGPT. It also seems likely this will never be fully accessible to the non-technical since I suspect any commercial applications will need to keep costs down and so minimize actually quite expensive API calls (executing a complicated gpt-4 summarization prompt across large text corpora for example). If you have the “data”, meaning of course text, and cost isn’t a concern, the results are astonishing and “lies” almost never a concern.

Honestly, it speaks mountains to Apple as an organisation that they did not invest all of their resources into Siri, especially when you consider how Amazon invested in Alexa etc. Everyone has long understood that the "personal assistant" represents really the pinnacle of not just what AI but the personal computer should be able to offer -- the mistake this past decade was thinking that had to do with then current ML paradigms or with these completely useless voice agents. That Apple could realise it was a dead-end and not sink much more investment/engineering resources into the tool any further than what anyone actually uses these tools for (changing song, lowering volume, etc) is outstanding.

Ultimately, I'm very reassured that it's Apple and Microsoft leading the way for personal and business AI assistants respectively. As Stratechery has emphasised, the latter not only already has all the data from most firms outside Silicon Valley, but more importantly has always adhered to and designed their software according to a computer-as-a-tool philosophy versus Google's "ML/AI will do everything for you behind the scenes"; the former has not only made a show of data security but invested massively in the hardware architectures necessary to making on-device AI a possibility. Without at all being a "fanboy", Apple is quite literally the only company I would trust my personal data to for use in a GPT-based assistant.

I’m part of the I’m sure infinitesimal and ever-shrinking portion of Amazon’s userbase who actually try to use the site to buy rare and used books. Most annoying for my experience is that they recently replaced what was in fact a best-in-class recommendation engine, which lead to a lot of my purchases, with a totally useless ad bar that displays universally terrible “related” books. There’s a world where after effectively cornering the world book market they subsequently destroy it (and they of course own and have effectively feature frozen competitors such as abebooks and bookdespository).

I share your review 1000x. I pretty much exclusively read PDFs, mostly OCRd but sometimes not. reMarkable is the only ereader I’ve ever found usable. I do agree that the subscription model feels tacked on, but there’s good tooling otherwise (I don’t use the cloud service myself). I highly recommend it as an ereader if you want one predominantly for research, actively taking notes on your documents at home and on-the-go while wanting to sync that with your PC note apparatus.

Anecdata of course: I've been a frequent and global user of Uber for far too long, but in the last six months I've unintentionally stopped using the app completely both locally and while traveling. At least in Europe, the Uber experience, for drivers and riders, has become egregious and competition from aggregators of local taxi infrastructure can finally offer a well-designed app at an almost identical price point with much faster pickup/better service -- and, of course, better comply with local employment regulations. Compound that with orders of magnitude smaller/cheaper engineering teams and I suspect we might be at a tipping point away from ride-sharing towards (aggregated) taxis; I will be watching FreeNow closely in the EU and would love to see their growth/financials.

Apologies, missed this response. Do you have an email address I can reach you at? It depends somewhat if you’re really interested in the “human” or “social” sciences, but I really do recommend that Eco How to Write a Thesis book regardless. It conveys the old secrets of how to enter humanistic scholarship — and was originally written to allow even working class Italians who might only have access to a library for an hour a week to do so.

My last response to you as you’re obviously not arguing in good faith. The sentence that offended you is very simple. PhD programs in the UK are almost universally dissertation-only. While full-time you are expected to finish in three to four years, you can enroll part-time and finish in eight. The latter path is intended for those who are self-funded and typically already working full-time in whatever profession. An MA or MSc, with as or more stringent coursework than in the US, would be required in either case, but the MA can also be completed part-time. Look at the website of almost any English PhD program.

Again, my only argument is that the EU offers more options for advanced studies alongside another career than the US, which is something I have firsthand experience with.

There are multiple paths, but the easiest is what is called an intra-company transfer. Basically after a year working for a company that has an office in the UK you can easily get a visa that will allow you to move overseas and work for that office, bringing family etc. As an engineer at even a medium-sized high growth startup, there are many opportunities to switch offices in this manner and I’d highly recommend it, even if you aren’t interested in studying.

In the UK at least, MA/MS degrees are largely autonomous from either PhD or BA programs. They indeed are predominantly coursework-based alongside a research component, but typically offer a part-time option that is entirely manageable alongside full-time work. It’s a very competitive market and quality varies wildly, but it’s often possible to do an MA there in a discipline you may not have studied as an undergraduate — I’ve known several people who, over their working life, picked up numerous post-graduate degrees in multiple disciplines (even for an international student, we’re talking about 5-6k per year part-time).

Yes, completely agreed with your corrections. Research is really the term I should have used and, as I clarified in a response above, it’s certainly true that, especially in STEM, there are at least industrial and academic tracks.

I would just clarify that my only intention was to address the particular case of someone currently working in tech who would like to additionally pursue non-STEM studies for their own sake. In that case, rare but that I wish were not so, the part-time, dissertation-only PhD programs of many European nations (Germany and the UK in particular) are, on average, a better fit.

I read your post in good faith and respond in kind. There are certainly very important details I did not elaborate in a quick post, but I assume you understand that the question under consideration is how someone pursuing a full-time career might partake in (non-STEM) higher academic degrees. While I regret using the expression “rest of the world”, I don’t believe it’s just hyperbole to say that this is easier in Europe.

You are correct that people pursue different paths after US PhDs. However, as you say, a US PhD contains heavy course work and (outside STEM, where I have few data points since most researchers I knew dropped out to work for startups/FAANG) the average full-time program duration is only going up. There are exceptions of course, but admissions are centered around a personal scholarly narrative rather than a particular project and pricing is generally under the expectation of grant funding. Again, there are further details we could write books about, but I don’t think this is far from a general picture of US PhDs in the humanities and social sciences.

In Europe, MAs and PhDs are separated as many have noted. I know most about the UK context, where MAs are affordable and can be taken part-time, completed in two years with one class a semester. PhDs, which are research only (I agree with the you and would indeed change “writing” to “research” if I wrote the post again), can also be done part-time over eight years for a very nominal cost (about 5k a year for an international student). Systems have become more standardized in the EU after the Bologna accords towards the more professionalized three year track, but I’ve known PhD students in Germany to take 10+ years writing their dissertations. Unless policies have changed in the past few years, the same goes for France.

Again, comparing post-graduate education between US and EU, let alone between EU countries, is a complex topic. I have attempted only to show that there is a substantial regional difference and that, at least outside STEM, the EU system is more accessible for those with full-time careers, who do not plan to work in academia.

I posted above before I'd read this -- I'd really stress again that, at least in the humanities, writing is the difference between mere 'accumulation of knowledge', which we all know will eventually be forgotten, and understanding. I would actually recommend Umberto Eco's book How to Write a Thesis as a great foundation for how to structure your cross-disciplinary studies and synthesize, again even if just for yourself. If you do start a blog, however 'cliched', you can also start to seek validation (or, even better, critical commentary!) from individuals in your fields of interest.

It feels very 'old internet' and his particular objects of study may not be of interest to you, but http://www.autodidactproject.org/ is the very interesting record of one man's lifelong studies outside the academy, as well as reflection on that process.

Best of luck! Certainly ignore those who say this is no different than watching TV if you feel a genuine 'conatus', i.e. striving and desire, for these pursuits.

I'm very similar and would just give you two pieces of advice: (1) buy books compulsively and don't do your research predominantly online, and (2) make sure you write often, whether or not you publish your work or keep it to yourself. I think the latter is the most important advice for those trying to pursue an intellectual path outside of academia -- if you don't develop a habit of writing, you aren't really synthesizing what you've learned and developing your thought. Don't worry if the quality is not great to begin with (and certainly don't worry that it seems cliched).

The other thing I would suggest, if your career and family would manage, is spending time working in the UK. It's very easy to get a visa if you're involved in the tech industry and their higher education system for MA/MS degrees and PhDs is very different than the US, both more affordable and better suited towards working life. In the US, the thought is that if you're doing a PhD, say, it's a full-time professionalized pursuit under the assumption you will be a professor; in the rest of the world, a PhD is essentially an apprenticeship in writing under a mentor for however long it takes you, often with no aim (i.e. career) other than that writing itself.

Personally, I've alternated between highly-paid years working in the tech industry and years studying or writing without employment to great personal satisfaction (and managed to help publish a few scholarly anthologies that wouldn't have happened without someone who didn't need to rely on academic grants etc).

I decided not to pursue a PhD in favour of working in tech and in exchange decided that I would never hesitate to buy any book that interested me. Over a decade, I've acquired 1000+ physical books, most of which are non-fiction. The internet is a poor substitute for (especially old) books, I've found.

For a more interesting story, look to the origins of the Warburg Library in London [0]: the eldest heir of a wealthy banking family passed on his considerable inheritance on the condition that his brother would purchase any books in any quantities that he wanted. This incredible and unclassifiable collection of 6000+ primary source books, which formed the basis of its owners' incredible and unclassifiable historical research, can still be consulted today [1].

[0] https://warburg.sas.ac.uk/library/about-library/library-aby-... [1] https://en.wikipedia.org/wiki/Aby_Warburg

Coincidentally, within the original 1880 study [0] that first grappled with the phenomena it was most highly-correlated with scientists. He concludes that: "scientific men as a class have feeble powers of visual representation... My own conclusion is, that an over-readiness to perceive clear mental pictures is antagonistic to the acquirement of habits of highly generalised and abstract thought."

[0] https://web.archive.org/web/20160416064610/http://psychclass...

I've discussed the topic of "aphantasia" (the inability to form mental images [0]) with programmers at various jobs as well as with non-developer friends. It's certainly anecdata, but it's always been curious to me that I've only ever heard engineers identify with this condition -- and more frequently than the understood likelihood of it within the general population would lead me to expect.

So, again, not a known neurological effect of programming, but an interesting correlation I've observed between many of the best programmers I've known and the condition of aphantasia.

[0] https://en.wikipedia.org/wiki/Aphantasia

In political science, the term populism has a long history of which I can only offer a small sketch. The paradigmatic cases are to be found in Latin American politics over the last century, especially the Argentinian leader Juan Peron [1]. In its classic form, the term is used to describe a wave of political parties who were neither pro-communist nor pro-capitalist, who found great success in countries that were both electorally democratic while also highly economically polarized. In countries that had traditionally been governed by a small cultural/economic/military elite, this new generation of political leaders began to win campaigns by vote-seeking from the disenfranchised through whatever means, typically through a combination of anti-status quo rhetoric and redistributive policies. This isn't just to suggest that they weren't "authentically" in favor of the poor or anything like that, but to understand how such a phenomenon can change the dynamics of democracies, where politicians are lead less by values (whether they be classically conservative, pro-market or socialist) than vote-seeking at any cost. It's also why we can find both "left-wing" and "right-wing" variants of populism, depending on exactly the types of rhetoric (anti-immigrant or anti-rich, for example) and policies (build the wall or tax the rich) that are employed.

The phenomenon is heavily connected with what is called "clientelism", ie where politics becomes ruled by relations of patronage. Imagine a situation where a particular political party literally pays for membership, with the expectation of a vote and to mobilize for certain events. Such structures are not uncommon in rural areas of developing countries, where essentially feudal relations (think the same person, who conveniently owns all the land and employs everyone in an area, has been the mayor of a town for three decades) have been converted into seemingly democratic structures [2].

[1] https://en.wikipedia.org/wiki/Peronism or https://en.wikipedia.org/wiki/Populism_in_Latin_America [2] https://u.osu.edu/laspoliticas/2016/10/30/clientelism-in-lat...

I took a similar path and, in my experience, it seems to in fact be more uncommon for someone to maintain a real passion for the day-to-day work of programming long-term. Which is to say I wouldn't worry, but try to focus on finding an interesting and enjoyable next move, whatever that may be; just know that you're young enough to take a few risks and that the job market for engineers is still such that you won't have difficulty getting back on that track should you so choose.

Certainly one of the more predictable paths is moving up into engineering management.

In my case, after experiencing a bit of burnout in my late 20s, I debated between going back to school for an MBA---another common route in this scenario---and trying to find a more customer-oriented role within an interesting technology company. I ended up deciding to join a late-stage startup in order to help found their Solutions Architecture team, which has been an absolutely terrific experience. I could imagine going back to an engineering role in time, but I've enjoyed and learned so much traveling the world to work with a wide range of companies across industries---just long enough to understand their business and (hopefully) help resolve their core difficulties...

Feel free to message me (account @gmail) if you're interested in hearing about my experience or of course if you're looking for a new role (https://stripe.com/jobs/positions/solutions-architect).

Sticking to a Western context, this is a pretty fundamental distinction between the US and the EU in the understanding (and, crucially, in the implementation/enforcement) of human rights.

I can't pretend to do justice to the long history of the concept, but we can at least say that for the latter, privacy has been considered an important human right since at least the UN declaration of 1948. This has been carried over into European law, see all the iterations on EU data protection laws. The UN statement is Article 12: "No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks."

For the US, this dimension of human rights did not deeply inform policy. Here discussion around a "right to privacy" really began in a different context with Brandeis and a right to be "left alone", largely meaning from the press. Many of the cases that inform privacy law in the US are oriented towards such scenarios and do not necessarily translate well to the context of data. See http://groups.csail.mit.edu/mac/classes/6.805/articles/priva.... There is rather a discussion on the accuracy of financial data about a person that stems from credit reporting.

The other area that would have to be discussed is of course wire-tapping laws, but leave that for another day... In sum, the question of a "right to privacy" has a long tangled history even just within the West, but is decidedly a thing in the EU.

"History is like a grandmother; it loves the younger grand-children. To the latecomers it gives not the bones but the marrow, while Western Europe has hurt her fingers badly in her attempts to break the bones."

Agreed. For me the appeal of a smart watch would be to minimize the amount of times I actually have to take out my phone, which is always a potential distraction. Smart watches are interesting to me if they reflect a trend towards more limited devices that help instead of hindering focus---rather than so many devices that do everything...

Lives of members of social groups can be different for historical reasons. Because of this, the best selection of features (as in, how we select and encode relevant aspects of a dataset for a particular problem) that one would need to use, as well as the correlations between them, may be different between different groups. The question is not whether there is fundamentally a difference between, in this case, racial groups vis-a-vis paying back loans (i.e. that the only feature required in your model would be "is member of this group"), but what are the traits of possibilities of life that have for whatever reason ended up leaving a quantifiable trace in our databases and their distribution within that group.

One hypothetical example: suppose that there existed a group G that was not able to go to the top n% of universities due to discrimination. Your company uses some rank of university attended as one of the features input to its favorite machine learning algorithm. However, the dataset you trained on excluded group G. Within this group, the best university individuals have been able to attend is X which is by definition not in the top n%. Had the algorithm been trained on this group it would have observed that school X is highly correlated with success in this group, even if not in the original training set used. As is, your ML system assigns a low probability to members of group G.

Issues like this will be hard to prevent. While that doesn't mean we shouldn't work hard to make real innovations in ML, I think the legal approach of a "right to explanation" as analyzed in http://arxiv.org/pdf/1606.08813v3.pdf and recently added to European law is regardless a helpful tool to ensure accountability.

Ignoring any other aspect of your comment, I would highly suggest you to do a bit more research on Hepatitis C before suggesting it only concerns "druggies" and those who "have sex with weirdos carrying diseases." In fact, a large portion of those currently impacted (and especially those who would now be in an acute condition where this drug would most likely be literally life-saving) contracted the disease through hospital blood transfusions before 1992 [1] when there was not a proper diagnostic test [2]. As such, it is a condition that very much cuts across the population and income levels.

[1] "Before 1992, when widespread screening of the blood supply began in the United States, Hepatitis C was also commonly spread through blood transfusions and organ transplants." - http://www.cdc.gov/hepatitis/hcv/cfaq.htm

[2] Though even that might be disputable per http://www.theguardian.com/society/2007/may/26/health.politi....

Seems like a more reliable metric would be take all nearby Yelp listings with no reviews and then remove any of those remaining that have any Foursquare/Instagram check-ins at all...

As a follow-up, I highly recommend http://monoskop.org/Computing_and_cybernetics_in_CEE#Episode... as a much more detailed account of the history of Soviet hardware and thought about computing, including even the relation to cybernetics.