HNHacker News
TopNewBestAskShowJobs

zhenjl

373 karma · joined May 28, 2013

submissionscomments
zhenjl··on Typing Chinese like English
Ok, trying to write 国家栋樑. Unfortunately I only got the first two characters..国 is Fqge, 家 is ei(, but haven't figured out how to do the last two. Any idea how to get the last two?

Quick suggestion: when shift key is not down, should show lower case letters. When shift key is down, show upper case letters.

zhenjl··on Typing Chinese like English
Ok, I figured it out...瓦 = qRre
zhenjl··on Typing Chinese like English
I really want to like this. Maybe what the developer can do is to build a simple app that translates a Chinese character into a series of keys. That will ease some of the frustrations on not being able to figure out how to type using this input method. For example, I am trying to type 瓦 but for the life of me I can't figure out how.

edit: removed "shift doesn't work"...does seem to work.

zhenjl··on Doudou Linux – For children from 2 to 12 years old
From http://www.doudoulinux.org/web/english/about/more-about/arti...

"“Doudou” is a French word that means wubby, the teddy bear or the cloth that children carry everywhere and hug very strongly in their arms before falling asleep. In China the word 豆豆 means the same and is pronounced the same too."

Chinese it's pronounced doughdough (4th tone, if anyone cares) as someone else mentioned below.

zhenjl··on Advice for New Managers
What a coincidence. HBR just ran a blog on How to Build a Strong Relationship with a New Boss, which seem complementary.

https://hbr.org/2016/10/how-to-build-a-strong-relationship-w...

zhenjl··on OweFS – One-way encrypted file system
Should call it 1fs with the number 1. 1fs.io is even avail!
zhenjl··on Sequence: A High Performance Sequential Semantic Log Parser at 175,000 MPS
Thanks for letting me know. I hadn't realized that. Will have to figure out why.
zhenjl··on Sequence: A High Performance Sequential Semantic Log Parser at 175,000 MPS
I don't have any first hand experience, but it seems like grok [might not be that performant](http://ghost.frodux.in/logstash-grok-speeds/)?
zhenjl··on Sequence: A High Performance Sequential Semantic Log Parser at 175,000 MPS
Thanks for the link. Do you have any info on the performance of this parser?
zhenjl··on What Happened to Old School NLP?
Try a static blog generator like hugo or octopress...
zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
I ran with GOMAXPROCS=2 on the command line. The post showed the actual commands that I used in each case.
zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384

Mac OS Yosemite..

zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
Ah..k...no peeking beyond just the head of the queue.
zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
Heh. No worries. Excellent list of questions. Like antirez, I am evernoting (is that a verb now?) this for future references.
zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
When I ran this I gave the server and client 2 cores each. Neither were maxing out the cores.

At about 856K MPS (sent + received) @ 1024 bytes/msg, that's less than 1 GBps. So I can't imagine it being io(net) bound either.

So inconclusive...unfortunately. Need to profile a bit more to know.

Edit: Tho this is TCP based so it's possible that it's netio bound given the protocol overhead. At this message rate, each packet is almost guaranteed to be full size. So we are doing about 600K pps, approximately.

zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
I have not. But I am going to assume you are telling me to learn more about how to calculate my performance numbers. :) Fair point and I will bookmark for future reading.
zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
Author here. The message size tested is 1024 bytes.
zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
Excellent question regarding latency.

Tyler Treat who wrote an excellent blog post on MQ performance tested this and got mean latency for 1000000 messages of 98.751015 ms.

https://github.com/tylertreat/mq-benchmarking/pull/5

zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
Author here. A bit of a pleasant surprise to wake up and see this on HN.

Thanks for the detailed response. Here's some answers hopefully can help clarify things a bit.

* Are messages delivered once and only once?

A: MQTT allows QoS 0 (at most once), 1 (at least once), and 2 (exactly once.) The performance numbers in the blog are for QoS 0.

However, SurgeMQ implements all three and there's unit tests for all three. I just haven't done the performance tests for QoS 1 and 2.

* Are messages are messages delivered at least once?

SurgeMQ supports it though the numbers posted are for QoS 0 (at most once)

* Can messages be dropped entirely under pressure? (aka best effort)

No. Currently no messages are dropped.

* If they drop under pressure, how is pressure measured? Is there a back-pressure mechanism?

See above.

* Can consumers and producers look into the queue, or is it totally opaque?

Not sure what this means..sorry..

* Is queueing unordered, FIFO or prioritised?

Ordered, FIFO...MQTT spec requires that messages from publishers to delivered in the same order to the subscribers.

* Is there a broker or no broker?

Brokered

* Does the broker own independent, named queues (topics, routes etc) or do producers and consumers need to coordinate their connections?

Brokers uses topics to route. Publisher publishes to a topic, subscribers subscribe to multiple topics w/ optional wild cards.

* Is queueing durable or ephemeral?

Ephemeral currently. Though MQTT spec requires that any unack'ed QoS 1 and 2 messages be redelivered when the server restarts or client reconnects. So once SurgeMQ meets that spec, it could be considered somewhat durable.

* Is durability achieved by writing every message to disk first, or by replicating messages across servers?

N/A

* Is queueing partially/totally consistent across a group of servers or divided up for maximal throughput?

Currently SurgeMQ is a single server w/ no clustering ability. However, MQTT spec does mention the bridge capability that is a poor man's cluster. Not yet implemented.

* Is message posting transactional?

QoS 1 and 2 starts to be more transactional. QoS 0 is strictly fire and forget.

* Is message receiving transactional?

See above.

* Do consumers block on receive or can they check for new messages?

Block on receive.

* Do producers block on send or can they check for queue fullness?

Block on send.

zhenjl··on SurgeMQ: MQTT Message Queue at 750,000 MPS
Author here.

There's really no comparison tbh. RabbitMQ has been battle tested for years and SurgeMQ is just at its infancy. And RabbitMQ has a lot more enterprise security features that SurgeMQ doesn't have today. So if you are looking for a solution today, go w/ RabbitMQ.

SurgeMQ hopefully will get there someday, but it's not ready today, yet.

zhenjl··on Another go at Go failed
Did you by chance profile the Go program to see where the hot spots are?
zhenjl··on Go vs Java: Decoding Billions of Integers Per Second
I did not. It might be difficult tho for all the delta encoding methods since they all depend on computing delta from the previous integer.

For the non-delta methods it is conceivable that ||ism may help, but not sure by how much.

zhenjl··on Go vs Java: Decoding Billions of Integers Per Second
2013-11-17 Update #2: Tried another new trick. This time I updated the leading bit position function, when using the gccgo compiler, to use libgcc’s __clzdi2 routine. This had the same effect as the update #1 except it’s for when gccgo is used. Performance increase ranged from 0% to 20% for encoding only. Thanks dgryski on reddit and minux on the golang-nuts mailing list.
zhenjl··on Go vs Java: Decoding Billions of Integers Per Second
Ok, I think I definitely fixed it now. Thanks for the heads up.
zhenjl··on Go vs Java: Decoding Billions of Integers Per Second
I just finished updating the code with an assembly version of a hot function (finding the leading "1" bit position) and updated the blog to reflect the improvements. In general I saw a 3-40% performance increase using the assembly version, which I borrowed from math/big/arith_*.s. The reason for the wide range is because it depended on the size of the delta integers being processed.
zhenjl··on Go vs Java: Decoding Billions of Integers Per Second
bsg75, thanks for this. I updated to serve images from the site instead of from Google. Hopefully this helps. Please let me know if you still run into problems.

I haven't updated the data section yet.

zhenjl··on Go vs Java: Decoding Billions of Integers Per Second
Interesting. Those are served from Google. Not sure what's going on there. Anyone else have this problem?
zhenjl··on Improving Cithhash Performance by Go Profiling
Thanks for the suggestion. I resubmitted here: https://news.ycombinator.com/item?id=6710115
zhenjl··on Improving Cithhash Performance by Go Profiling
Thanks for the heads up. Wow...can't believe I missed that. Will have to resubmit.
zhenjl··on Benchmarking Integer Compression in Go
Sorry for the late reply. I was traveling for the past week or so.

In any case, EWAH is designed to avoid the WAH patent. According to the author, the Apache people checked things out when they adopted it for Apache Hive and they were apparently satisfied.

If Apache is satisfied, I think we are safe. :)

Page 1 of 2Next →