Let's investigate this!
For reference, a common electricity mix currently generates about 0.5 tons of CO2 per MWh [0][1].
Now, for the device in question:
Creating 1 ton of cement releases about 1 ton of CO2 [2]. A common ratio of cement per mass of concrete is maybe 1/5 [4]. The article says they found a way to reduce that to a sixth, so we are at about 1/30 of the concrete mass in cement. From the article, each block weighs 35 tons. So we'll just assume 1 ton of cement (and CO2) per block.
The graphic in the article shows ~40 layers of no more than 20x6 concrete blocks can be stacked around the tower, each weighing 35 tons. Therefore, each such installation would cost about 5000 tons of cement or CO2.
So its building cost in cement alone is the equivalent of 10000 MWh. Fully "charged", it stores 20 MWh. It would therefore have to complete 500 full cycles of taking energy available for free (and otherwise lost) and feed it back. Roughly assuming it can do that every week (no idea, really depends), that would be ten years to become CO2 neutral, just in terms of cement. So there's your reference value.
However, I haven't seen this kind of calculation for other energy sources. That would be interesting.
Edit: Here is a review of CO2 from lithium-ion batteries:
https://www.ivl.se/download/18.5922281715bdaebede9559/149604...
They say 150-200 tons of CO2-equivalent (!) per MWh stored, which is very close to the device in the article (in the above estimate it would be 250 tons of CO2 per MWh, although I did not check whether that is pure or CO2 equivalent).
[0] https://www.eia.gov/electricity/state/ (randomly sampled some states)
[1] https://www.umweltbundesamt.de/sites/default/files/medien/37... (German statistics)
[2] https://blogs.ei.columbia.edu/2012/05/09/emissions-from-the-...
[3] https://sans10400.co.za/concrete-mixes-by-weight/