EmanueleMinotto

I considered it, but for some reasons I didn't do it:

1. the main closure is useless if the .php file isn't included manually using require(_once)/include(_once) functions ( https://github.com/EmanueleMinotto/crystal/wiki/Tip ) because it returns directly at line 144 2. the autoloader become useless too, because there's Composer that does it very well 3. the PHP world is full of great frameworks, microframeworks, routers and other components, it will become rapidly a common library and not a good library because it doesn't follow development best practices but only a set of rules (defined in the CONTRIBUTING.md file)

Crystal can't be packed in a Composer package, but Composer can be used in a project including crystal.

It's more an experiment than a real microframework, it's absolutely not recommended for production. The contributing page should sound like a challenge :) https://github.com/EmanueleMinotto/crystal/wiki/Contributing

You're right, I forgot to write it explicitly in the article but if someone will follow istructions (extract all <a> tags and add them to the index) that method is tacit.

If it's not included in robots.txt rules and doesn't have a canonical link that's not a problem, because the bot can't know if those pages are different or not so those pages you linked are different. This is the reason why crawlers can't try to fill forms.

If you are really sure that these pages are the same, try checking the body content (if two or more pages have the same MD5 of the content, those pages are the same) or look for a form that generate those URLs.

A NoSQL solution is good because it's a DBMS (allows you to order collections). :) Filesystem is not good because you would need to order links' files in visited descendent order (not allowed in much filesystems), and to check if an URL is in the index you must store it with the MD5 as file name. A small DBMS like SQLite is not good for obvious reasons.