mingodad

For those that want to explore the grammars listed at https://github.com/tree-sitter/tree-sitter/wiki/List-of-pars... in a more friendly railroad diagram format I made https://mingodad.github.io/plgh/json2ebnf.html that reads the "src/grammar.json" and try it's best to generate an EBNF understood by (IPV6) https://www.bottlecaps.de/rr/ui or (IPV4) https://rr.red-dove.com/ui where we get a nice navigable railroad diagram (see https://github.com/GuntherRademacher/rr for offline usage).

I've just update the EBNF for rarilroad diagram here https://github.com/google/zetasql/issues/134#issuecomment-14... with the full grammar.

There is also https://melt.cs.umn.edu/ that has an extension that add templates and algebraic data types to C : https://github.com/melt-umn/ableC-template-algebraic-data-ty...

There is also https://github.com/ondras/wwwsqldesigner :

WWW SQL Designer, your online SQL diagramming tool

For people interested on grammars there is also:

- Convert tree-sitter grammar.json to EBNF https://meimporta.eu/tree-sitter/json2ebnf.html

- An alternative syntax to write tree-sitter grammars https://mingodad.github.io/lua-wasm-playground/ (select "Tree-sitter-ebnf-generator" from "Examples" then click "Run" to see the tree-sitter grammar for the content in "Input Text (arg[1])")

- An Yacc/Lex online editor/tester with several example grammars https://mingodad.github.io/parsertl-playground/playground/

Looking at https://llvm.org/docs/TableGen/ProgRef.html I've just added a partially working grammar/parser for tablegen in https://mingodad.github.io/parsertl-playground/playground/ an Yacc/Lex compatible online editor/tester (select "Tablegen-LLVM parser (partially working)" from "Examples" then click "Parse" to see a parser tree for the content in "Input source").

Any feedback is welcome !

There is also gram_grep[0] "Search text using a grammar, lexer, or straight regex. Chain searches for greater refinement."

See also parsertl-playground[1] for online edit/test grammars.

[0]https://github.com/BenHanson/gram_grep

[1]https://mingodad.github.io/parsertl-playground/playground/

I also just add a JQ parser/grammar to the online LALR(1)/FLEX grammar editor/tester at https://mingodad.github.io/parsertl-playground/playground/ select "Jq parser (partially working)" from examples then click "Parse" to see a parser tree of the source in "Input source".

Any feedback is welcome !

It's not true "the best chance you have is reading the 14 thousand-line parse.y file and trying to understand it", there are several tools to navigate yacc/bnf style grammars like https://www.bottlecaps.de/convert/ and it's companion https://www.bottlecaps.de/convert/ that make relatively easy to understand/document/debug/compare the grammar.

Just added several ruby grammars here https://github.com/mingodad/plgh/tree/main/ruby they are converted (mainly using https://www.bottlecaps.de/convert/) to an EBNF understood by https://bottlecaps.de/rr/ui to generate navigable railroad diagrams.

Copy and paste the EBNF on https://www.bottlecaps.de/rr/ui on the tab "Edit Grammar" the click on the tab "View Diagram" to see/download a navigable railroad diagram.

You can also try it online here https://mingodad.github.io/CG-SQL-Lua-playground/ using the Lua language.

This seem to be a public repository for it https://github.com/UoYCS-plasma/GP2

Just in case someone want's to try it I've did some work to allow build on linux (Ubuntu 18.04 and probably others too), you can see it here https://github.com/mingodad/cyclone .

On linux chromium 7.5GB of memory and 100% of one cpu.

It seems too hack depending on "__attribute__((always_inline))", tcmalloc for performance, specific libunwind to dump backtraces, doesn't build with g++.

Maybe the general idea is valuable but the implementation ?

I did something like this here https://github.com/mingodad/lalr-parser-test where I expanded byacc/bison/lemon to generate a naked grammar, ebnf grammar (understood by https://www.bottlecaps.de/rr/ui to generate railroad diagrams) and interchange the grammar between then (byacc/bison to lemon and the other way around too, taking in account the difference in how they interpret rules precedence).

It's an interesting approach to bring more expressiveness to C, this part need a bit more thought:

"If any spaces or parentheses appear in the type name, they will be substituted by an underscore. Thus unsigned int will become unsigned_int the type BitInt(27) becomes BitInt_27_ ."

Because "unsigned_int" (or any combination of the base types with an underscore) can already be used on user code.

Also to add to your list is https://github.com/KuramitsuLab/pegpy it's an intriguing peg parser with origin from a Japanese group https://github.com/nez-peg/nez .

To add to your list I just found this project https://github.com/ChrisHixon/chpeg it has good performance and it's relatively simple compared to others.

Also I recently discovered https://github.com/mingodad/lwc lightweight c++ preprocessor and here are some conversation and links about it https://github.com/mingodad/cfront-3/issues/6 and also an ASK HN link https://news.ycombinator.com/item?id=30511113

Here https://github.com/mingodad/cyclone is my attempt to update the latest available cyclone code and fix it to build on more recents operating systems.

The first step is build as 32 bits as clean as possible then attempt to update to be able to also build for 64 bits.

Actually it's already working on Ubuntu 18.04 see the build here https://github.com/mingodad/cyclone/actions .

One of the main reasons to do it is to preserve for historical purposes .

There is also this other repositories that seems to try something similar:

https://github.com/moon-chilled/cyclonic https://github.com/catb0t/cyclone https://github.com/pippijn/cyclone https://github.com/iphydf/cyclone

Would be nice if mozilla/linux/... foundation give funding to it and made it opensource and part of OS's like libc is right now.

Remark: It's for research and historical purpose to have a working version of CFRONT.

I would appreciate if someone have or know where to find the missing versions based on http://www.softwarepreservation.org/projects/c_plus_plus/ind... and the readme at https://github.com/junchih/cfront/blob/master/cfront_3/READM... the missing versions are:

CFRONT 3.0

CFRONT 3.0.1

CFRONT 3.0.2

With then and the change log the understanding of the source code would be a bit easier.

Here is a sample to test the "pedantic" patch:

====

select "23 % 3", 23 % 3;

select "12.3 % 3", 12.3 % 3;

select "12 % 2.5", 12 % 2.5;

select "23 / 3", 23 / 3;

select "12.3 / 3", 12.3 / 3;

select "12 / 2.5", 12 / 2.5;

create table ta(a text, b integer, c float);

select 'insert declared types == value types';

insert into ta(a,b,c) values('a', 1, 2.0);

select 'insert declared types != value types text';

insert into ta(a,b,c) values('b', '1', '2.0');

select 'insert declared types != value types';

insert into ta(a,b,c) values('c', 1.0, 2);

select 'update declared types == value types';

update ta set a = 'a' ,b = 1, c = 2.0 where a = 'a';

select 'update declared types != value types text';

update ta set a = 'a' ,b = '1', c = '2.0' where a = 'a';

select 'update declared types != value types';

update ta set a = 'a' ,b = 1.0, c = 2 where a = 'a';

select 'update one value declared types != value types';

update ta set b = 1.0 where a = 'a';

select 'update one value declared types != value types';

update ta set a = 49 where a = 'b';

====

Output default sqlite:

====

23 % 3|2

12.3 % 3|0.0

12 % 2.5|0.0

23 / 3|7

12.3 / 3|4.1

12 / 2.5|4.8

insert declared types == value types

insert declared types != value types text

insert declared types != value types

update declared types == value types

update declared types != value types text

update declared types != value types

update one value declared types != value types

update one value declared types != value types

====

Output of sqlite with "pedantic":

====

23 % 3|2

FP Remainder received non integer values 3.000000 :: 12.300000

12.3 % 3|0.0

FP Remainder received non integer values 2.500000 :: 12.000000

12 % 2.5|0.0

23 / 3|7

FP Division received non integer values 3.000000 :: 12.300000

12.3 / 3|4.1

FP Division received non integer values 2.500000 :: 12.000000

12 / 2.5|4.8

insert declared types == value types

insert declared types != value types text

Affinity applied on make record 1 : 1 : D

Affinity applied on make record 2 : 1 : E

insert declared types != value types

Affinity applied on make record 1 : 2 : D

update declared types == value types

update declared types != value types text

Affinity applied on make record 1 : 1 : D

Affinity applied on make record 2 : 1 : E

update declared types != value types

Affinity applied on make record 1 : 2 : D

update one value declared types != value types

Affinity applied on make record 1 : 2 : D

update one value declared types != value types

Affinity applied on make record 0 : 3 : B

====

I also proposed once something like "PRAGMA pedantic" where any type conversion would be reported to stderr see the changes and a sample output:

https://sqlite.org/forum/forumpost/692ae69237f2537b?t=h

https://sqlite.org/forum/forumpost/103d7294464fcde8?t=h

Thank you for explaining your understand of the issue presented here with a wide view.

I was shocked when I got the results shown here and thought of times in the past were I copied and pasted formulas (and sometimes did some small changes to be accepted by the language I was using) without thinking that any language would interpret it differently without warning.

Other good worth replies in my opinion:

amichal https://news.ycombinator.com/item?id=28100036

hinkley https://news.ycombinator.com/item?id=28101243

hansvm https://news.ycombinator.com/item?id=28100156

I believe that most of us are acting in good faith most of the time and we need some common examples/tests like basic math expressions that we could use to check/evaluate any/most implementations in programming languages to try prevent or minimize divergences like the ones shown here by a random generated expression that has no other meaning than a uncommon valid? test case to show an issue.

After removing all modulo operations and transforming all numbers to floating point:

====

EXPR = (((((((788.0)(8.46))))+8342.01.803-1.0))4186.4(15.0))(((22.0/((7530.0/((2.0)(((((25.0))-421))))))*597.0)+7283.8-9.60+167.0))+(8871.0)

SQUILU = 8259816920615.1

LUA = 8259816920615.1

JAVASCRIPT = 8259816920615.111

SQLITE = 8259816920615.11

POSTGRESQL = 8259816920615.11377654520912550160000

MYSQL = 8259816920615.113

AMPL = 8.25982e+12

PHP = 8259816920615.1

RUBY = 8259816920615.111

PYTHON = 8.25981692062e+12

====

Using https://www.onlinegdb.com/online_fortran_compiler

FORTRAN = 8.51528699E+12

====

Program Hello

Print *, (((((((788)*(8.46))))+8342*1.803-1))*4186.4*(15))*(((22/((7530/((2)*(((((25))-421))))))*597)+7283.8-9.60+167))+(8871)

End Program Hello

====

And after removing all modulo operations:

====

EXPR = (((((((788)*(8.46))))+8342*1.803-1))*4186.4*(15))*(((22/((7530/((2)*(((((25))-421))))))*597)+7283.8-9.60+167))+(8871)

SQUILU = 8515287402650.3

SQLITE = 8515287402650.34

POSTGRESQL = 8515287402650.347200

D = 8.51529e+12

JAVA = 8.515287402650345E12

MYSQL = 8259816920501.086

JAVASCRIPT = 8259816920615.111

PHP = 8259816920615.1

LUA = 8259816920615.1

AMPL = 8.25982e+12

GAMS = 8.25982E+12

RUBY = 7701542593121.873

PYTHON = 7.70154259312e+12

====

After looking for a text generator for grammars I found this https://github.com/dmbaturin/bnfgen and then to try understand the underlying algorithm I came out with a C simple program for a simple expr grammar see it here https://github.com/dmbaturin/bnfgen/issues/2#issuecomment-89... and also a thread on sqlite forum here https://sqlite.org/forum/forumpost/6885cf9e21

And making all operands to modulo operator "%" integer:

=====

EXPR = (((((((788)*(8.46))))+8342*1.803-1))*4186.4*(15))*(((22%284/((7530/((2)*(((((25))-421))))))*597%2663)+7283.8-9.60+167%((3))))+(8871)

D = 8.29038e+12

JAVA = 8.290383058308305E12

MYSQL = 8034912576159.046

POSTGRESQL = 8290383058308.307200

SQLITE = 8290383058308.3

SQUILU = 8291561284461.3

JAVASCRIPT = 8034912576273.071

PHP = 8035491468054

AMPL = 8.03491e+12

GAMS = 8.03491E+12

PYTHON = 1.11064580608e+13

RUBY = 11106458060797.121

LUA = 11664732388290

====