Isinlor

The deal signed on Feb 21st 2014 says:

1. Within 48 hours after the signing of this agreement will be adopted, signed and promulgated a special law that will restore the action of the Constitution of 2004, as amended by this time. The signatories declare their intention to create a coalition and form a national unity government within 10 days thereafter.

On February 22, 2014 as allowed by Article 111 of the Ukrainian Constitution, Verkhovna Rada decided to impeach Yanukovich with 328 votes for, 0 against and 122 not present or abstaining.

What do you mean by "intervened"? That they talked to people? Sure, they did talk to people.

Yes. Westerners and Russians seems to think that everything is organized by USA and that people of other nations do not have any autonomy.

A lot of people seem to think that Poland or Baltic states were somehow forced into NATO. Certainly narrative that China is trying to sell right now.

We knew very well that aligning ourselves with Russia is just straight road to disaster. And we needed to get as far as possible as soon as possible from Russian imperialism when we had a small window of opportunity.

Ukraine was split on it and it did result in disaster.

The protests were peaceful at the very begging in November 2013. Then protests were dispersed and the spiral of violence escalation started culminating in death of around 100 protesters and less than 20 police officers in February 2014.

Towards the end it was certainly not a peaceful ordeal. It was more of a small uprising.

I recall snipers shooting to protestors.

But I don't recall RPGs being used by either side when I was following it. I could not find any mention of it right now. Any sources of that claim?

Yes - USA was involved, as was Russia, France, Poland and Germany.

I agree that discussing who should and should not be in the government of an independent state is wrong.

But there was no coup and Yanukovych himself signed an agreement whose first point stipulated government change, Yanukovych decided to flee on his own and Yanukovych was impeached by democratically elected parliament.

USA did not start the protests. The protests started because people, especially around Kyiv, wanted and still want more than ever across the whole Ukraine to join EU. Yanukovych was elected with the promise of joining EU as a strategic goal.

At the same time, on 27 February 2014, masked Russian troops without insignia took over the Supreme Council (parliament) of Crimea and captured strategic sites across Crimea. Crimea was annexed on 18 March 2014 by Russia.

Nobody in the west organized Euromaidan and it was not a coup. Viktor Yanukovych decided to abandon the country on his own without any military intervention.

The reason for Euromaidan is simple. Ukraine is poor and EU represents the road to prosperity. In 1991, after collapse of Soviet Union, Polish and Ukrainian economies were exactly the same size. The same GDP. Poland made very quick pivot to the west. We decided to join NATO in 1999 and EU in 2004. Last year Polish economy was 3 times bigger than Ukrainian.

Nothing good ever comes from the east.

The poorest people will be hit the hardest. EU and USA will find it relatively easier to adjust given our location and overall wealth.

We are nowhere near exaggerating it.

We should be simple executing what was agreed in the Paris Agreement as it reflects our best evaluation of the cost to benefits ratio.

But current policies are aiming at 1.5 degree higher than what we evaluated as the optimal solution.

At some point we will be "replaced". When you get AI to be able to navigate all user interfaces, communicate with other agents, plan long term and execute short term, we will no longer be the main drivers of economical growth.

At some point AI will become as powerful as companies.

And then AI will be able to sustain positive feedback loop of creating more powerful company like ecosystems that will create even more powerful ecosystems. This process will be fundamentally limited by available power and the sun can provide a lot of power. Eventually AI will be able to support space economy and then the only limit will be the universe.

From Polish perspective I'm extremely grateful to my parents generation for starting the collapse of the Soviet sphere and very quick and decisive pivot to the west. Especially, when compared to Ukraine. Poland and Ukraine had the same size economy in 1991. Last year Polish economy was 3 times bigger than Ukraine. Personally, I can travel, study and work wherever I want in EU with zero trouble. I was born in Poland, studied in Belgium and Netherlands, work in Belgium and live in Finland.

Russia is a terrible partner. Russia sympathizer in Ukraine brought destruction on that country from Russia. It's really tragic that Russia is obliterating people that considered themselves Russians living in Ukraine.

There is zero comparison between modern western projects and Russia imperialism that did not change in the last 300 years.

SpaceX is also using commodity hardware, but they duplicate it 3x with majority vote for error correction.

SpaceX works around that by using consumer grade stuff and just replicating it 3 times. Tesla also managed to handle issues with supply shortages very well.

The old practices are just not good enough.

I don't know what kind of accountability do you expect? Presidents who started these wars are no longer presidents. Also, Americans wanted and supported wars in Iraq and Afghanistan. Even withdrawal from Afghanistan was politically costful for Biden.

Digital Markets Act give very broad executive powers to EU Commission to make sure that it will get implemented.

Specifically:

- article 7: Compliance with obligations for gatekeepers

- article 10: Updating obligations for gatekeepers and

- article 11: Anti-circumvention

Fines are up to 10% of annual global turnover, or daily fines up to 5% of average daily annual global turnover.

In more detail:

Article 11, Anti-circumvention

1. A gatekeeper shall ensure that the obligations of Articles 5 and 6 are fully and effectively complied with. While the obligations of Articles 5 and 6 apply in respect of core platform services designated pursuant to Article 3, their implementation shall not be undermined by any behaviour of the undertaking to which the gatekeeper belongs, regardless of whether this behaviour is of a contractual, commercial, technical or any other nature.

2. Where consent for collecting and processing of personal data is required to ensure compliance with this Regulation, a gatekeeper shall take the necessary steps to either enable business users to directly obtain the required consent to their processing, where required under Regulation (EU) 2016/679 and Directive 2002/58/EC, or to comply with Union data protection and privacy rules and principles in other ways including by providing business users with duly anonymised data where appropriate. The gatekeeper shall not make the obtaining of this consent by the business user more burdensome than for its own services.

3. A gatekeeper shall not degrade the conditions or quality of any of the core platform services provided to business users or end users who avail themselves of the rights or choices laid down in Articles 5 and 6, or make the exercise of those rights or choices unduly difficult.

Digital Markets Act give very broad executive powers to EU Commission to make sure that it will get implemented.

They don't need to change law to address issues.

Specifically:

- article 7: Compliance with obligations for gatekeepers

- article 10: Updating obligations for gatekeepers and

- article 11: Anti-circumvention

You have it backwards. Feeding people with meat is difficult.

This is why meat is expensive and why people in less developed countries can't afford as much meat as Americans.

If we were faced with famine even army could provide sufficient logistical support.

The only problem during COVID was that it was not profitable. It was cheaper to dispose it than to try to sell it.

Why it isn't suitable for humans? To me it seems perfectly fine.

It is certainly not the best food source, but it a subsistence.

During famines in Finland people used to eat even bark, especially inner bark.

https://en.wikipedia.org/wiki/Bark_bread

I don't think there is any difference from operational security perspective.

Russians would know very well that MiGs are flying from Poland no matter whether they would go trough Germany or not.

And they would have to track them precisely in order to attack in both cases as well.

The stupid part was going with the MiGs transfer public in the first place back at the beginning of the war.

But I recall it came from EU official and Ukrainians, not from Polish government directly.

There is "war in return". It is far from certain that Putin is willing to start nuclear war. In my view, it is unlikely that he would use nuclear weapons. And if he is willing to use them he will use them because he is loosing in Ukraine, without regard as to why is he loosing in Ukraine.

They can not and must not be revoked. This will allow Russia to build up its army again. Give them 5 maybe 10 years and they will start a new war. We must keep Russia isolated for as long as they are a fascist state. If Russia gives up Putin to be trailed in the international court for war crimes then we can lift the sanctions, not earlier.

As Pole - finally!

> In particular, any hypothesis that tends towards magick, for example suggesting that a change in quantities (data, compute, training time) yields qualitative improvements (prediction transmogrifying into understanding, overfitting transforming into generalisation), should be discarded with extreme prejudice.

It does not tend towards magic. It does happen and people can replicate it. Melanie Mitchell recently brought back the point by Drew McDermott that AI people tend to use wishful mnemonic. Words like understanding or generalisation can easily be just a wishful mnemonic. I fully agree with that.

But the fact remains. A model that has ~100% training accuracy and ~0% validation accuracy on simple but non-trivial dataset is able to reach ~100% training and ~100% validation accuracy.

> This hints to the fact that the effect depends on the structure of the dataset and so it's unlikely to, well, generalise to data that cannot be strictly controlled.

Indeed, but it is still interesting. It may be that it manifests itself because there is a very simple rule underlying the dataset and the dataset is finite. But it also seems to work under some degree of noise and that's encouraging.

For example, the fact that it may help study connection of wide, flat local-minima and generalization is encouraging.

> You are impressed by the fact that one particular, counter-intuitive result was obtained

I'm impressed by double descent phenomenon as well. And this one shows up all over the place.

> There is a well-known paper by John Ioannidis on cognitive biases in medical research: Why most published research findings are false

I know about John Ioannidis. I was writing and thinking a lot about replication crisis in science in general. BTW - it's quite a pity that Ioannidis himself started selecting data towards his thesis with regard to COVID-19.

> It's not about machine learning per sé, but its observations can be applied to any field where empirical studies are common, like machine learning.

Unfortunately, it applies to theoretical findings too. For example, universal approximation theorem, no free lunch theorem or incompleteness theorems, are widely misunderstood. There are also countless less known theoretical results that are similarly misunderstood.

> And that suffices to explain its performance, without the need for any mysterious post-overfitting grokking ability.

It actually still does not suffice. It is just not expected no matter what the authors would be doing.

Just the fact that they managed to get that effect is interesting.

Granted, the phenomenon may be limited in scope. For example, on ImageNet it may require ridiculously long time scales. But maybe there is some underlying reason we can exploit to get to grokking faster.

It's basically all in fig 2.:

- they use 3 random seeds per result

- they show results for 12 different simple algorithmic datasets

- they evaluate 12 different combinations of hyperparameters

- for each hyperparameters combination they use 10+ different ratios of train to validation splits

So they do some 10*12*3*2 = 720 runs.

They conclude that hyperparameters are important. Seems like weight decay is especially important for the grokking phenomenon to happen when model has access to low ratio of training data.

Also, at least 2 other people managed to replicate that results:

https://twitter.com/sea_snell/status/1461344037504380931

https://twitter.com/lieberum_t/status/1480779426535288834

One hypothesis may be that models are just biased to randomly stumble upon wide, flat local minima. And wide, flat local minima generalize well.

The paper shows that their model first overfitted the data. By overfitting I mean 100% train dataset accuracy and ~0% validation dataset accuracy. The model never gets any feedback from the validation dataset trough the training procedure.

Everyone's expectation would be that this is it. The model is overfitted, so it is useless. The model is as good as a hash map, 0 generalization ability.

The paper provides empirical, factual evidence that as you continue training there is still something happening in the model. After the model memorized the whole training dataset and while it still has not received any feedback information from the validation dataset, it starts to figure out how to solve validation dataset.

Mind you, this is not interpretation, this is factual. Long after 100% overfitting, the model is able to keep increasing its accuracy on dataset it has not seen.

It's as we discovered that water can flow upwards.

Grokking was discovered by someone forgetting to turn off their computer.

Nobody knows why. So, nobody is able to make any theoretical deductions about it.

But I agree that fig 3. requires interpretation. By itself it does not say a lot, but similar structures appear in other models like in the one where we discuss elements sequence prediction. To me, the models figure out some underlying structure of the problem, and we are able to interpret that structure.

I tend to look at it from Bayesian perspective. This type of evidence increases my belief that the models are learning what I would call semantics. It's a separate line of evidence from looking at benchmark results. Here we can get a glimpse at how some models may be doing some simple predictions and it does not look like memorization.

I still don't know what you mean by n ≤ 10.

Have you played around with the demo?

http://recur-env.eba-rm3fchmn.us-east-2.elasticbeanstalk.com...

You can input there a sequence and it will predict the expression and next elements.

For example, for the sequence: [3, 1, 8, 9, 18, 19, 37, 74, 92, 166, 332, 406, 738, 1476]

It predicts: u_n = u_(n−1) + (u_(n−3) (n mod 3)) + u_(n−4)

It's a really decent guess, and if I give it one more element (from 14 to 15 elements) it gets it correctly.

Yes.

Basically there are two approaches to sequence prediction.

The traditional style, linear regression, ARIMA, RNNs etc. where you directly predict the next element in a sequence. The output is on the same level of abstraction as the internal values used in the model.

There is also the new-ish style where you predict symbols instead of predicting the values directly. You can predict symbols representing numbers or you can also predict a symbolic formula that can be used to extrapolate the values perfectly. This is the way humans do it.

And my point is that when you look at the symbols embedding they do have interpretable structure that model can use to generalize. And experiments seems to suggest that DNNs models are indeed generalizing.

If you look at the 3 sequences I gave you, can you guess following elements of the sequence?

We can create sequences, but guessing underlying patterns is a lot more difficult.

Humans will have very hard time if you go beyond around 10 operators in a pattern used to generate a sequence.

My guess is that their model will be better at it than me or you.

I don't think you will find any human that will extrapolate sequence generated with more than 10 operators. And longer input sequences are actually easier to handle - fig 1. the right most graph.

If you think you can do better than their program then:

Seq1: [0, 1, 2, 3, 6, 7, 13, 26, 32, 58, 116, 142, 258, 516]

Seq2: [2, 2, 3, 5, 10, 12, 22, 44, 54, 98, 196, 240, 436, 872]

Seq3: [3, 1, 8, 9, 18, 19, 37, 74, 92, 166, 332, 406, 738, 1476]

Their program is able to guess correct continuation with one more sequence element.

SHA1 hash for verification: bef5e213340f91258b3b9a0042c9c083dd91cb80