For anyone with Long Covid, there is an online support group with a lot of resources available (it does use Slack). The group can be found at: https://www.wearebodypolitic.com/covid-19
342 karma · joined April 29, 2009
Ok, lets say you have two large tables: Sentences and Annotations, and you make a view that joins those, lets call it vwSentencesDetailed. Then you do a query like:
select * from vwSentencesDetailed where srcID = _srcID
Where srcID is an indexed field in the Sentences table that will let you get a narrow subset of the data. Let's say the particular file it came from....
The performance characteristic I observed was that the it behaved like (loosely):
select * from (Select * from Sentences s Inner Join Annotations a on a.sentID = s.sentID) vw where srcID = _srcID
rather than like:
select * from (Select * from Sentences s Inner Join Annotations a on a.sentID = s.sentID where srcID = _srcID) vw
I believe that SQL Server makes such a performance optimization, but Postgres does not.
Having the right syntax for databases is critical. Here's the moderately optimized SQL for two DB systems for two different updates:
SQL Server
Update tok Set spid = est.sid FROM @Tok tok INNER JOIN @eST est ON tok.cs = est.cs
For SQL Server -- using the table alias for the update is critical for good performance. Another trick for SQL Server can be to do something like (which is recommended by the SQL Server team -- and it yields excellent speed improvements):
Update tok Set spid = est.sid FROM (SELECT * FROM @Tok) tok INNER JOIN @eST est ON tok.cs = est.cs
Postgres
Update _Tok Set spid = est . sid FROM _Tok tok INNER JOIN _eST est ON tok.cs = est.cs WHERE tok.cs = _Tok.cs
For Posgres, the performance critical line is the addition of the WHERE clause to bind the Update table to the FROM table. It makes sense -- once you understand (from the SQL Server perspective) that it is equivalent to using the alias in the Update clause
For my part, I thought Postgres was slow until 5 months ago (when I came from a port from SQL Server).
Then I learned something! The default install of Postgres is slow (at least on Windows). And on benchmarks against the baseline SQL Server port I had, it was about 2x slower. BUT there were a number of things I could do to make it faster: a) Use the EnterpriseDB wizard for tuning Postgres --> resulted in a 2x to 3x performance improvement. b) Adjust the way I did updates (there is a way to get Postgres updates to be fast, it just takes some massaging) c) Get rid of views when possible -- I found that certain query compilation optimizations I had taken for granted on SQL Server weren't available.
Doing all this resulted in a system that performed faster and w/ less RAM than SQL Server(and I had spent a bunch of time tuning the SQL Server perf in the first place!)
Since 2.0 VB.Net has started falling behind, for example not getting the "yield" convenience that C# got. So on the MS provided languages, C# is continously getting better than VB.Net, and you'll hear even less advocacy for VB.Net!
That said, the .Net framework can be used with more interesting languages (my preferred one is a python-like language called Cobra: www.cobra-language.com) and the primary loss in the equation is the Visual Studio environment (although MS is making it easier to write language extensions in 2010 version). So you can use an excellent framework with many other languages.
So then what you are left with is the language itself. I can't speak for others, but the script-like syntaxes speak to my brain more naturally than C-like languages, and I have found that I am most effective in thinking in a language like python than I am with either VB.Net or C#.
Part of the appeal that python/ruby have is they typically come with the non-Windows OS a person might use. For instance Python comes out of the box with OS X and most *nix OSes.
You are right on the abstracting what you've learned. It is true for a general language family, and less so (but still helpful) when you cross families. By that, if you stay in the procedural/OO realm of languages, you'll notice lots of similarities, but if you branch out into functional languages it will be very different.
But that was not my "rep"...
The initial draw is that it is a python-like language for .Net (important since my day job is in .Net). And while I've dabbled with Boo and IronPython, Cobra seems to have removed most of the syntactic worts that I found w/ Python, as well as a few other features.
In particular, as someone who hasn't really done automated unit testing or contractual programming, I'm drawn to the fact that support for unit testing and contracts is built in. I like the python-like syntax as I find it easiest to learn new programming concepts in (which I can then generalize to my day-to-day work).
"On the contrary: the programmer should let correctness proof and program grow hand in hand. Argument three is essentially based on the following observation. If one first asks oneself what the structure of a convincing proof would be and, having found this, then constructs a program satisfying this proof's requirements, then these correctness concerns turn out to be a very effective heuristic guidance."
Anyone else think that TDD is a close approximation of what Dijkstra was talking about in the quote? FWIW - I say this as a non-practioner of TDD.
ReadyBoost is different from PreFetch (they are features you can turn off/on independent of each other) which I believe you are referring to when you mention "pre-loading". The wikipedia article is decent: http://en.wikipedia.org/wiki/ReadyBoost
Really, this would be: 35,000 mph / (60 minutes per hour) / (60 seconds per minute) = 9.72 miles per second.
Then just decrease the distance traveled by a factor of 60, so instead of 386 billion miles traveled by V2, it would be about 6.4 billion miles.
The interesting factor was when they took brand identity away as a criteria given to the graders of the translation Bing and Yahoo's quality scores rose.
I once contacted Cuil about some worthless search results, and got a standard reply asking me to be patient since they were a small company. But there wasn't any hint in the email that they would actually address the issue, so drastically reduced my use of them and never bothered to contact them again. Listening to users would have probably helped if my experience indicates a pattern.
My suggestion as to what the scientist is supposed to do when encountering a completely unscientific argument is below. Please note, I am not trying to respond to any of the other aspects or questions you asked -- this "simple" response became way too long as it is :-)
PREPARE AHEAD OF TIME: A. Learn to communicate. A lot of life is dealing with miscommunication. It seems scientists as a group have the same issue with miscommunication that software developers generally have. This is very important where the science is critical for impacting public policy decisions. Especially, learn to listen, so that the metaphors and analogies that are guiding the incorrect thinking can be ascertained.
B. Expect that as soon as you are using science to advocate for public policy changes scientists will have opposition. Actually, the science part isn't the prereq, it is the "advocate for public policy changes". Lots of groups who advocate for public policy changes get villianized. It sucks, but it should not come as surprise to anyone who enters into that arena. If you (generic) want to be left alone, keep your head low. If you (generic) want to stand up for something -- you will be opposed.
REACT: C. Educate. In my opinion, the scientist is supposed to educate people in a simple a terms as possible. I believe scientists used to view this as part of their duty and I believe it has been something that has been generally lost (fwiw I have a 3 volume set of articles that great scientists such as Einstein and Bohr wrote for lay people that span over two centuries... often the articles were published in newspapers.. I have loaned it out, otherwise, I'd list the set for those interested). To educate the lay people, write articles for news papers or online venues. Engage critics with openness, because in doing so professionalism is shown and confidence is built up.
D. Be respectful. Don't refer to people who disagree as deniers or anti-science or ignorant. Give them the benefit of the doubt. Figure out the faulty metaphors they are employing and reveal them. At the worst, agree to disagree. I have recently encountered a scientist doing exactly this, and have been very impressed with him (and I expect I disagree with him on some fundamental beliefs), but I have contacted him to encourage him in what he is doing, and even sought his feedback on some of my own interests.. because I can trust him to be methodical AND respectful. Respect opens up the possibility for long term dialogue. Also, keep in mind that your audience isn't only the person asking questions or postulating absurd theories. It includes the people who are undecided watching the debate (or reading the news article, etc..) as well as the minority / subset of the opposing view who are willing to listen.
FOLLOW UP: E. Make a point of gathering objections, alternative theories etc, and having the data that refutes them readily and publicly available. This is just a tactical confidence building move.
SUMMARY: Please note, I'm not encouraging necessarily that a single person do all these things. Happily scientists do not work alone, are part of associations, etc.. and collectively they can do these steps and maintain/gain/improve a good footing with the public. And doing so in a respectful way will gain more good will (even from people who disagree) than you might imagine.