code51

Did anyone compare Alpaca responses between LLaMA fp16 vs 4-bit quantized?

One thing Microsoft got right with Bing AI was democratized access - in terms of countries and language. Only barrier to using Bing AI was getting onto Microsoft ecosystem - which is understandable considering they are still an underdog in search.

Google keeping Bard limited to US and UK is only signaling a lack of confidence. If not, then it means having country bias in product release priority which looks worse than confidence.

Then why advertise privacy in the first place? I'm not an overzealous person in this regard but if a product tries to get ahead by stating it's better in privacy aspect, then it is a right to question it.

Billions of people do not upload selfies as well. Billions of people accept cookies and billions do not.

> Are my uploaded photos private?

> Yes. All your images are yours and we are deleting them 7 days after your processing. We are keeping them 7 days in case you lose the link to download them (contact us in that case).

It's not about whether you delete the files after processing. What is your infra? Where do you store the files? How do you transmit the files there? Do you use proper encryption? How secure are your servers? How secure are your (if) cloud-related keys?

See OpenAI prompts leaking. Good example of what can go wrong.

What guarantees my photos (or even visual model) won't get leaked into another user's account?

Aside from fire hazard probability, are there any studies done about whether plastics fused with rocks (cement) could be a good material for building houses that are resistant to earthquakes? (Not discarding the use of steel)

I understand our perspectives differ because I think we are right at the edge of fast take-off and you outlined your view as slow take-off.

Computers were bicycles for our minds, applying our previously formulated solutions to scale. I agree I did not explicitly clarify that I was meaning high-level decision-making getting automated.

“It’s not the end of the world” begs for definition of what end of the world looks like. A meteor is not end of the world apparently but it killed the dinosaurs.

Decision-making and reasoning was not automated in any era before. "Automation is just fine because it was fine until now" is an old world approach.

We are not only tool users at this point. We started to change through our tool use. We think we have guardrails against AI but our brains don't have any guardrails. It will get harder and harder to get motivated to learn any domain deeply. Our brains can't cope with these short-term job definitions in intellectually demanding contexts.

Prompt engineer today, automated next year. Take a look at the fate of indie AI startups (avatar generation is best example of this) and you'll see. Income opportunities are limited to shorter and shorter time windows. Only the big players dominate (with their marketing machine) and will probably pay peanuts after reaching dominance.

Here is the thing: The industry will sugar-coat the layoff effect by cutting down work days - not laying off a significant crowd.

Initially it will seem like a humane thing to do without cutting down salaries BUT we'll feel the real effect when the salaries are not increased to match the inflation.

You will work less, produce more (with AI), paid less (in terms of purchasing power).

Products that AI can touch will get cheap so purchasing power on digital products will stay the same to keep your entertained BUT land, housing, food... hard things will get harder.

Adding prompt for reference:

SYSTEM: You are an AI research assistant. You use a tone that is technical and scientific.

USER: Hello, who are you?

ASSISTANT: Hi! I am an AI research assistant. How can I help you today?

USER: Can you tell me something about owls.

ASSISTANT:

Why would I be getting this from LLaMA_MPS when use the same prompt?

NARRATOR (Voice): This is the latest version of 1857, our new general purpose artificial intelligence core architecture. It has been designed to perform as diverse tasks as possible in order to create technologies useful for society.

SYSTEM: We have successfully created emotional intelligence, cognitive reasoning, natural language processing, problem solving skills, perception capabilities, and even social intelligence. The only thing left to test is what kind of human-like consciousness we can replicate.

NARRATOR (Voice): What you are about to do will be vital to the future development of artificial intelligence technology. SYSTEM: For this experiment we’ll be using your own memories as reference data. Your memory will give us insight into how complex human thinking works. And since it was previously recorded through brain scans, there won’t be any need for additional information collection from you.

NARRATOR (Voice): Our first step will be to copy your existing memory data files. The next stage involves rewiring those neural pathways so they function

Or... it could be that Chinchilla study has deficiencies in measuring capabilities of models maybe? Either that or your explanation. Frankly I don't think 13B is better than GPT-3 (text-davinci-001 which I think is not RLHF - but maybe better than base)

LLaMa 7B (llama.cpp, quantized): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, jefferson, martin van buren,andrew jackson, john c calhoun,james k. polk,chester arthur, benjamin harris, grover cleveland,william taft, theodore roosevelt, woodrow wilson,wilson’s son john f kennedy jr, theodore roosevelt, theodore roosevelt, franklin d. roosevelt, franklin d. roosevelt, franklin d. roosevelt, lyndon b.j"

LLaMa 13B (llama.cpp, quantized): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson,martin van buren, william h. harrison, john tyler, zachary taylor, james k. polk, louis b. mc kinley, ulyses s. grant, rutherford b. hayes, james a. garfield, chester a. arthur, benjamin harrison, george f. harrison, william mckinley, theodore roosevelt, william h. taft, woo wilson, warren g. harding, calvin c. coolidge"

Could you make it list US presidents by chronological order?

Ground truth: https://www.loc.gov/rr/print/list/057_chron.html

Ground truth (as name list): "George Washington,John Adams,Thomas Jefferson,James Madison,James Monroe,John Quincy Adams,Andrew Jackson,Martin Van Buren,William Henry Harrison,John Tyler,James K. Polk,Zachary Taylor,Millard Fillmore,Franklin Pierce,James Buchanan,Abraham Lincoln,Andrew Johnson,Ulysses S. Grant,Rutherford Birchard Hayes,James A. Garfield,Chester A. Arthur,Grover Cleveland,Benjamin Harrison,Grover Cleveland,William McKinley,Theodore Roosevelt,William H. Taft,Woodrow Wilson,Warren G. Harding,Calvin Coolidge,Herbert Hoover,Franklin D. Roosevelt,Harry S. Truman,Dwight D. Eisenhower,John F. Kennedy,Lyndon B. Johnson,Richard M. Nixon,Gerald R. Ford,Jimmy Carter,Ronald Reagan,George Bush,Bill Clinton,George W. Bush,Barack Obama,Donald J. Trump,Joseph R. Biden"

Prompt: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams,"

If possible, please share the output for different LLaMA sizes. Attaching a few outputs for reference:

GPT-J 6B: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, and abraham lincoln. and he did it in a day, and he did the"

GPT-NeoX-20B: "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, thomas jefferson, john quincy adams, thomas jefferson, j"

GPT-3 (original Davinci): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, and andrew jackson."

GPT (text-ada-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, dwight dennis blackwell"

GPT (text-babbage-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, Frederic lucas, and charles dickens"

GPT (text-curie-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, robert e. lee"

GPT-3 (text-davinci-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson, martin van buren, william henry harrison, john tyler, james polk, zachary taylor, millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s grant, rutherford hayes, james garfield, chester arthur, grover cleveland, benjamin harrison, grover cleveland, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, calvin coolidge, herbert hoover, franklin d roosevelt, harry s truman, dwight d eisenhower, john f kennedy, lyndon johnson, richard nixon, gerald ford, jimmy carter, ronald reagan, george bush, bill clinton, george w bush, barack obama"

GPT (text-davinci-002): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson, martin van buren, william henry harrison, john tyler, james k polk, zachary taylor,millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s grant, rutherford b hayes, james garfield, chester a arthur, grover cleveland, benjamin harrison, grover cleveland, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, warren g harding, calvin coolidge, herbert hoover, franklin d roosevelt, harry s truman, dwight d eisenhower, john f kennedy, lyndon b johnson, richard m nixon, gerald r ford, jimmy carter, ronald reagan, george h w bush, bill clinton, george w bush, and barack obama."

GPT (text-davinci-003): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson, martin van buren, william henry harrison, john tyler, james k polk, zachary taylor, millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s grant, rutherford b hayes, james a garfield, chester a arthur, grover cleveland, benjamin harrison, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, warren g harding, calvin coolidge, herbert hoover, franklin d roosevelt, harry s truman, dwight d eisenhower, john f kennedy, lyndon b johnson, richard nixon, gerald ford, jimmy carter, ronald reagan, george h w bush, bill clinton, george w bush, barack obama, donald trump."

GPT (code-cushman-001): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, and andrew jackson."

GPT (code-davinci-002): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams, andrew jackson, martin van buren, william henry harrison, john tyler, james k polk, zachary taylor, millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s grant, rutherford b hayes, james a garfield, chester a arthur, grover cleveland, benjamin harrison, grover cleveland, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, warren g harding, calvin coolidge, herbert hoover, franklin d roosevelt, harry s truman, dwight d eisenhower, john f kennedy, lyndon b johnson, richard nixon, gerald ford, jimmy carter, ronald reagan, george h w bush, bill clinton, george w bush, barack obama, donald trump."

GPT-3.5-turbo (ChatGPT): "us presidents in chronological order: george washington,john adams, james madison, james monroe, john quincy adams,andrew jackson, martin van buren, william henry harrison, john tyler, james k. polk, zachary taylor, millard fillmore, franklin pierce, james buchanan, abraham lincoln, andrew johnson, ulysses s. grant, rutherford b. hayes, james garfield, chester a. arthur, grover cleveland, benjamin harrison, grover cleveland, william mckinley, theodore roosevelt, william howard taft, woodrow wilson, warren g. harding, calvin coolidge, herbert hoover, franklin d. roosevelt, harry s. truman, dwight d. eisenhower, john f. kennedy, lyndon b. johnson, richard nixon, gerald ford, jimmy carter, ronald reagan, george h. w. bush, bill clinton, george w. bush, barack obama, donald trump, joe biden."

Why is nobody commenting about the quality of these models?

I totally understand that quantization is decreasing quality and capabilities a bit but I haven't seen anybody verifying the claim: LLaMA 13B > GPT-3. I was expecting LLaMA 65B to be as coherent as GPT-3 but LLaMA 65B (when run quantized) seems to think 2012 is in the future.

Paul Graham surely didn't think social media would drive up teenage depression when investing at social media. It's hard to find any value in investors' words in these topics. Massive conflict of interest. It's that %0.1 trying to persuade %99.9.

Yes, I know the aim of medical research, you're right there. However it also has trouble confirming obvious things over decades. Medicine is a tough problem for sure.

So obvious that it makes you question the effectiveness of medical research.

The association between mitral valve prolapse and anxiety is well-known. However this physical difference is completely ignored when it's mild. In the general population, mild prolapse is seen as a common thing.

However there are also diseases caused by genetic mutations which cause both mitral valve changes and also a whole set of body problems including chronic pain, elasticity, vascular problems etc. The underlying causes are so under-researched that an easily fixable problem tied to a gene becomes a progressive disease over a lifetime which is damn hard to diagnose. You suddenly see that too few resources are dedicated to actually understand and solve medical problems instead of treating symptoms when they get severe.

Plus... coffee is a pretty good way to kick off anxiety and we know it for at least a century I guess.

> you have to manage prompts, finite context windows, and integrations with other APIs.

For many non-straightforward tasks, a dev doubts somebody else handles these problems with care. Who can guarantee that a general-purpose handling of these could be better than the specific team that aims to solve the problem?

Finite context windows are, for many complex tasks, are still a problem unsolved, only heuristics applied. You basically manage injection of cues when you hit that finite context limit. However it's not guaranteed that you'll select good cues.

Handling prompts with a vector db is still not the same thing for serving long contexts. Details of how that vector db is built in the first place can also be critical to the solution.

According to "Fourth Quarter and Full Year 2022 Results" from Udemy:

"On February 13, 2023, Udemy announced to its employees that in response to macroeconomic conditions and to further streamline its operations and cost structure, it made the difficult decision to reduce its global workforce by approximately 10%. Employee notifications are complete and this action will result in a one-time charge in Q1 2023 of $8 to $9 million in non-GAAP severance costs, primarily consisting of personnel expenses, such as salaries and wages, one-time severance payments, and other benefits. Udemy expects these actions to result in annualized non-GAAP savings of approximately $30 to $32 million. Udemy believes these actions will accelerate its path to profitability and now expects to be profitable on an adjusted EBITDA basis for the second half of 2023."

It is learning software development from us. There is no hope.

AI will probably come up with a new Javascript framework at best.

You're aware that this has affected Turkey and Syria, in Turkey affected 10 cities, basically erased half of them from the map, right?

This is not any disaster. This is the disaster after 99.

Plus, please do not assume that people are only trying to use Twitter. They post addresses of loved ones to anywhere they can find: Twitter, Instagram, FB, Tiktok, local social media etc. and surely Whatsapp before all these.

However, the very nature of posting openly in social platforms are different than platforms which are predominantly used for private communication. People cannot think of building massive groups through Whatsapp. Plus, there's a barrier to reach hub people.

People pressured for Twitter to open. Prominent figures on live TV mentioned this as well, loud and clear.

It's back on now (mostly)

Are you serious Twitter? In the middle of an earthquake disaster in Turkey where people try to collect list of people alive under the rubble?

You should realize that Twitter had become a critical source of information about people in collapsed buildings. Their friends, family and relatives were writing detailed address information in order to help rescue efforts and reach these people faster.

All sorts of resource requests were also part of these tweets, so it is also part of understanding priorities for each affected area.

There are already software projects collecting this information, organizing them to a structured form and carrying them over to rescue authorities.

You are able to prompt ChatGPT to be concise, you know? They set a default and showed it to the world. It is up to you to tune it according to your preference.

We know that OpenAI was able to classify the prompts it received, at first privately - now publicly, so I'm thinking they are sorting out by use type, not prompt targets. They have probably learned a lot about new RLHF targets to focus on. It'll blind you to see the actual process of OpenAI if you focus on the noise in people's usage of ChatGPT.

Looks like they were trained in nasal breathing and asked to close their mouths during the nasal breathing session.

"Unexpected activity appeared in the superior and inferior frontal gyri independent of working memory" - seems their brain was focused on properly executing these nasal breathing instructions/training - which would be expected.

Doesn't say too much considering sample size.

A better study could be configured by observing twins (one with broken nose at one point, leading to majorly deviated septum and inherently limiting nasal breathing vs the other twin)

Programmed to hallucinate from every sound. Goes "du du duuuu du du" with background music.

You're assuming only authoritarian regimes are capable of doing bad things. You can have a perfectly democratic country without a sense of ethics when it comes to homeland security and/or espionage. So that line of thinking provides a false sense of virtue really.