HNHacker News
TopNewBestAskShowJobs

jhancock

3,661 karma · joined May 8, 2008

submissionscomments
jhancock··on The One-Person Framework in Practice
Excellent write up.

I worked with Rails and Phoenix in their early days and got plenty of value from each. If you're building a traditional web 2 app, look no further...similar to choosing Postgres, start there until you have really good reason to venture off.

Without taking away anything from these frameworks and as someone that spent over 10 years building app frameworks, sometimes it's not what I want.

I'm using Clojure for my current problem space which would stymy me if I tried to use Rails or Phoenix. I spent the past 4 months doing product/domain "shaping". There are no web pages yet..mostly pure server side domain and API calls for data gathering. After this exploration I now have several working subsystems and have figured out the pathway to the mvp which will come together quickly. As a bonus I have a working domain core to leverage for steps after the mvp.

jhancock··on Dipping my toes in OpenBSD, in Amsterdam
thanks for the good read.

I used Amsterdam BSD two years back to scratch my itch. It was a no nonsense, perhaps perfect way to try out OpenBSD.

The thing keeping me from using OpenBSD/FreeBSD on my new production app is I don't know what kinds of issues I may run into compared to Debian stable on a fairly beefy dedicated server.

My new app has dependancies:

a) http reverse proxy. No problem

b) Java 21 / Clojure 1.12. JDK 21 virtual threads is a must. Unclear what level of support/quality can be expected on OpenBSD or FreeBSD.

c) Postgres 17 (TimeScale)

jhancock··on Bayard: a full-text search and indexing server written in Rust
I'm looking for an easy to use typeahead/autocomplete search solution. javascript lib for frontend paired with easy to manage, lightweight server. something modern.

The dataset isn't huge. e.g. 1 million strings of no more than 512 utf-8 chars each and not reindexed more than once a day or week. clusters, sharding etc unnecessary.

I keep hoping to stumble on a fully baked solution...any ideas?

jhancock··on Swift 4.0 Released
As a new swift'er working on an app with my 12 year old son, I reasonate with your comment. We were near the end of our first app and hit a need in swift 3 to figure out locale/language settings so needed to manipulate strings. I was shocked at how unobvious it seemed. Fortunately, I found a swift playground cheat sheet and left it to my son to explore. A couple hours later he had it worked out ;)
jhancock··on Google Drive issue affecting a significant subset of users
Excellent post. Thanks for taking the time for such a thoughtful reply.

For those reading along or for my own purposes of tracking thoughts for a case study of this experience, I'll add some comments:

> Drive isn't (at least yet) designed with a central group share in mind.

Trouble is Google advertise it as being suited for this. Google provides well written docs to guide you through setting up a central file share for your organization. I just looked for the doc again. Its been moved/removed in the last few weeks. Now I see they have a new/improved feature called Team Drive which may solve some aspects of my troubles.

> Especially if they use one of those insane massive shares that the whole company has write access to.

Google advertises a 10 terabyte drive for $99 a month. The Business and Education plans advertise unlimited storage. This combined with a wealth of other docs Google provides for using G Suite for Business may lead a reasonable person to conclude it could handle a measly 1.5 terabyte drive account.

> Drive API: I may be misunderstanding your use case, but I think this is just as simple as logging in as the records user and authorizing the application instead of granting it global scope to the GSuite account.

Yeah, that's the next step for us to write some code against. Google Drive's API docs tell you that for server-to-server API usage, use the service account method. This method gives access (for a given scope) to all accounts. I've found no docs or example code for server to server API using the client/server OAuth method which would restrict access to a single account. rclone uses the client/server OAuth method. Its tricky stuff. To do this on a server we need to write a process so a server admin can manually bootstrap the OAuth access and cache the access tokens for reuse by the server. This method requires a bit of tricky work to refresh the server's access with a refresh token at times. No docs on how long this access is good for or how often you can refresh. So to do this for long running server access, we would need some method so the server can email me when the token expires so I can manually bootstrap the OAuth again.

> records@mydomain makes me think you're using a file sharing system for something that would be better served by purpose built software.

Sure. My use case wan't clear in my original post. The reason I picked this approach is Google Docs is great for our needs. Having the student's work evidence embedded in a shared drive (in read only folders) alongside other folders containing Google Docs of teachers assessment, feedback and admin docs is pretty slick. Actually this stuff is working great. Its the ops/admin aspects that aren't ready for business.

> Sync client: If bandwidth is constrained roll out the client to users in comfortably small increments.

When I checked about a month again, my only option as an admin was on for all users or off for all users. Hard to manage a roll out plan when Google pushes their sync tool on users and I can't control access on a per account basis. For now, its off. I'm happy to leave it that way ;)

jhancock··on Google Drive issue affecting a significant subset of users
Thanks much. Good to know I'm not alone or imagining this stuff ;)

> * Your API issues with jpg files are probably due to you tripping Google's anti-piracy system.

Holy smokes! Didn't know this was baked in. Our school teaches art and creativity. The jpg's being uploaded are pictures of student's original work: painting, metal works, jewelry, glass blowing, hand written diaries and sketches. Our images are pretty darn unique (and some amazing art as well). Would love to know if Google's detectors are tripping up.

> * Set up a separate Google Domain for testing stuff. You'll have to pay extra for it, but don't test on production data.

Good idea. That solves part of the problem. But it doesn't enable me to have a set of test data based on near-current production data. And god forbid I someday make a mistake on production...no way to rollback.

jhancock··on Google Drive issue affecting a significant subset of users
Interesting. Since experiencing these troubles with Google Drive, I have been wondering how green the Microsoft grass may be ;)

The key reasons we made the decision to migrate from our internal file systems were:

* The old internal system is dying and out of space. So either reinvest or move to the cloud.

* We're a school. Half our teachers work remote as do around 70% of our students. The old system meant no students had direct access to their records and remote staff had to use a clunky and slow Remote Desktop over VPN solution.

* Our new records management solution for students and staff lives in the cloud. It seemed appropriate to integrate to the Google Docs/Drive solution in the cloud. Integrating into an internal file server would have been tough.

Anyways, something something about the best laid plans ;)

jhancock··on Google Drive issue affecting a significant subset of users
Its been a rough few months. I thought I had done significant due diligence last year. I read lots of docs, we wrote a bunch of test code and released test fileshare accounts to staff, had around 8 months of trial usage internally before cutting over.

Nothing prepared me for the reality of production.

Google Docs is doing well. Its Drive that seems not "ready for business". And Google's tech support is very coy. That don't acknowledge these problems when I get them on the phone.

jhancock··on Google Drive issue affecting a significant subset of users
Thanks for your thoughtful reply. I'll give /r/sysadmin a go.

Google tech support recommends posting to Stack Overflow and tagging it so their support folks can take notice. So far in my reading of lots of SO posts, the Google tech support official answers are of little help.

jhancock··on Google Drive issue affecting a significant subset of users
Apologies for the long post...this is a bit of a cry for help...is there a support group for programmers/admins that have been bitten by the Drive cool aide?

My organisation did a migration to Google Drive (from internal Windows server) in the last few months. I've had quite a few issues (I'm the administrator).

I'm posting this (abridged) laundry list of problems here...would love to have some feedback or pointers to what's wrong or what I'm doing wrong. When I've managed to get through to Google Drive tech support I don't get much in the way of answers.

For context, I'll confine this to just two central share accounts: fileshare@mydomain and records@mydomain. Each account has around 1.5 terabytes of files. Our org has 85 G-suite accounts (users). This is a small to mid sized business.

* I'll mostly skip the part of how difficult, slow and expensive a process it was to get around 3 terabytes into Google Drive. rclone was a good friend to us. It took months as we had to constantly re-sync as we migrate....we are running a business while we migrate. I'll also mostly skip the problems we had with the Drive sync client...its default behavior is to sync everything. I had to globally disable usage of this tool. Guess what happens to your company network when 50 users install the sync tool and click "add to drive" for the company fileshare. Good luck educating users with a wide range of PC skills on proper sync usage.

* Drive's file sharing operations are opaque and take an indeterminate amount of time to complete a share (could take days). If I need to restrict access to a large subset of files, I have no idea when the share is complete. I have found that sometimes it does not complete for all users and I'm left with some files not shared. I can't find a pattern. I have no tools to help me find what (under a large set of folders) didn't get shared. If a user or me as admin needs to unshare, this can take days. So there is no practical way to protect sensitive access files except put them in different root folders or different share account.

* No practical way to get a backup. I'm trying spanning backup at the moment. Its been running against my fileshare account all week. The spanning backup console tells me I have about 100 days to wait to complete the initial backup. Yeah, 100 days to backup 1.5 terabytes. What's the use of unlimited storage if I can't protect it? In this case, I'm suspecting Google Drive, not Spanning as I've become familiar with how performance variant the Google Drive API is.

* Google Drive allows an end user to easily do "destructive" things to the central file share. Last Friday I received a call from a staff. She knew something bad just happened and quickly reached out for support. She had accidentally used the move or delete operation in the Drive UI to cut out a huge subset of our fileshare. I say "move or delete" because she didn't know and as an admin I couldn't figure out what happened. I dug around both the fileshare account and her account and couldn't find the files. The Trash folder on both accounts was empty. This didn't make sense as the activity panel clearly showed her and other users deleting things. At first the Trash view returned a server error. I refreshed my browser and got it to show. Empty. I tracked through the activity list and found the operation of concern....it showed the folder had been removed. Fortunately, I was able to click on the folder in the activity panel and it took me to it. From what I could tell, the files were in the system, but not attached to anything...no parent folder, not marked as deleted or moved. I was able to move this unattached folder under a new temp folder at the root of the fileshare account. Then go to that folder and move it again back to its original place. Then we wait...for the removed share permissions to fall back into place. Good thing this happened late on Friday as we had the weekend to wait for the shares to come back.

* The Google Drive API is slow and unstable. I'm a programmer. We spent quite a bit of time writing a system to push files from our records management system to our Google Drive records account. Currently, we are adding less than 1000 files a day; mostly mid sized jpg's (1 to 3mb). So well below our API limit of 1 billion requests a day. Our server using the drive API tracks errors. Currently, around 1/3 of the uploads fail and need to be retried up to 10 times. Some never make it and we have to handle it manually. Good thing we wrote this in Elixir/Erlang as fault tolerance on the API client is our friend.

* The Google Drive API is "global". When you create a service account (Google's API method for server to server communication), you give access for the scope of the API but not the account. Our system needs read/write access to only the records account. No can do. You have to give your API keys full read/write access to every account. So yeah, your dev and test systems have full access to all production data. Better be a very careful programmer. If you mess something up in dev, you could mess up your production drive accounts.

* Combine these API problems with the inability to do a backup (and restore as well as restore to a test account) in any reasonable amount of time...and you can throw away your discipline of separating dev from production and being able to run repeatable tests before deployment.

I'm going to spend the rest of my day reviewing code with a programmer. The code is to refactor our folder structure in our records account. It shouldn't delete anything...just move lots of folders into a new structure. But if we have just one bug in our migration scripts, well who knows. So I'll be reviewing code today and on pins and needles Saturday while our migration script runs on our production drive.

jhancock··on Barack Obama: Why we must rethink solitary confinement
The U.S. Constitution is structured in traditional contract form. It starts with stating its purpose (preamble) and continues with rules to support methods of execution of purpose.

It is not unreasonable that a U.S. President may consider these recent statements as fitting to "...promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity..."

jhancock··on Google services set for 'return' to China
Here's one long running anecdote. I've been operating a self-published book site for over five years. All Chinese content. Some of the books are banned in China. Much of the content no China online publisher will touch.

Baidu loves my site. Searches are often in the top three results on Baidu. I have no ads and pay Baidu nothing. Don't even help them find my site. Baidu even runs its own book site and gives my site plenty of love. To my knowledge the site has never been blocked by the GFW.

Google barely knows I exist even though I follow all their webmaster rules. I'm not sure if its because of low page rank or Google not being great at Chinese content.

I suppose my point is page rank isn't the only way to go or at least having a strong reliance on page rank may leave out unique or rare content that isn't marketed (quality inbound links) heavily.

jhancock··on Smaller Overseas Entrepreneurs Find China Too Frustrating
I think if you reflect on "contracts mean nothing and guanxi with people in power is everything" you may find the seed to an answer.

China has 1.4 billion people. There are many that will follow a contract or work with you to adjust agreements to both parties' mutual benefit.

If you are leveraging guanxi, ask yourself why you feel it is ok to further empower a person that is causing harm to society. There are many in China that do represent your values. When you engage in guanxi, you are "feeding the dragon": empowering the wrong people and dis-empowering the right people.

note: when I say "you", I don't mean you personally.

jhancock··on Smaller Overseas Entrepreneurs Find China Too Frustrating
Thanks for your level headed post.

I see too many Western news articles on China that present an us versus them attitude. Its unhealthy as commentary (if not the articles themselves) often contain undercurrents or direct statements of nationalism/racism.

In my experience trustworthy and competent employees are not any more difficult to find in China as the U.S. It does take work (and the frustrations stem from different places) but not in aggregate harder than in the U.S. Never had a bad employee in China. Never had to fire someone. Never had to discipline someone. I followed our written labor contracts and so did they.

As to people not following a written contract. It is true that you need to choose your partners carefully and manage your relationship as a written contract can often be ignored. That said, in ten years of running companies in Shanghai, every landlord I had for home and office was meticulous about following the contract. Never had a problem. I only spent four years in Chengdu and had fewer home and office leases but again, no problems.

jhancock··on With Obi Smartphones, John Sculley Takes Old Lessons From Apple and Pepsi
I'll put in a plug for Fairphone 2 https://www.fairphone.com/

There is so much to love about what these folks are doing.

jhancock··on Storage for Photographers, Part 2
Hello SmugMug CEO, I've received many emails the last two months about my credit card expiration. The problem is I cancelled my account a couple years ago. I have not gone through a couple of years of bank records to find when/if you kept charging me and didn't send an email.

I liked your service and cancelled because I was in China and couldn't reliably use the service. Every one of those reminder emails I've receive makes me feel like SmugMug is not a good company. I am not in China anymore and don't think I would use the service of a company that can manage to send so many emails to remind me they can't take money out of my bank but don't do so when they do.

jhancock··on China’s Internet Boom Starts to Fade
This article is mostly about stock market numbers. Ok, its from Bloomberg.

I don't think its useful to use words like boom and fade when your writing about companies that have been doing well for 10+ years: Alibaba, Sina/Weibo, Tencent, Baidu, Tudou.

Hard to say what kind of future Sina/Weibo and video streaming companies Youku Tudou could have...I guess similar to Twitter and Youtube. Alibaba and Tencent could have a good shot at being world leading banks over the next 10 years among the things they already do well. The stock market may not see its fair shake on these deals, but users in and out of China are getting benefits.

China's Internet companies are growing fine from my perspective as a consumer of online services and as a person building companies here. Software development and management gets better year over year. In the IT space, there seems to be organic evolution of treating employees better. More people are practicing taking risk in creating companies.

jhancock··on Learning Smalltalk from “Pharo by Example”
Last year I added a touch of programming to my 9 year old son's home school activities. We tried several learning tools, starting with Scratch. I don't have anything negative to say about the various options we tried. However, the most positive exercise was him going through the "Prof Stef" tutorial built into Pharo. He really got into exploring and interacting with objects using tradition Smalltalk inspectors and workspaces (Do it, Show it, Inspect it). He loved that he could close the environment and open it the next day with all his play right where it was.

As a father, I loved that I didn't feel like a fraud as I could easily help him dig around and answer just about any question on what's really going on.

jhancock··on What's all this fuss about Erlang? (2007)
Seems to me erlang adoption has been chugging along just fine (with or without HN posts). There are some high-profile startups using erlang: WhatsApp and I think Facebook Chat is/was.

There has been progress in cleaning up syntax, library and package management cruft that have helped ease adoption. Elixir (a new lang built on erlang) seems to be coming along well.

For most types of webapps, erlang has all the batteries included. The problem is that for many mainstream types of webapps, getting v1 out the door quickly in Rails/Django works well. I think that may be part of the adoption problem...if the problem domain doesn't clearly require erlang to get v1 out the door, many will use Rails/Django and never look back.

jhancock··on Dyad: Minimal, portable async networking library for C
yeah...epoll and kqueue in my experience are easily interchangeable. I built a server on FreeBSD and the port to Linux was straightforward. My first event-based socket usage was on Windows NT around 1999. When we ported the server parts to Linux, replacing with epoll was also straightforward.
jhancock··on Stripe: Alipay support
three questions:

1 - Will you handle micropayments (i.e. not rake 30 cents on top of 3%)? Alipay excels at this.

2 - Can some/all of the RMB sent to the Alipay account not get exchanged to USD and auto transferred to the Stripe account holder's bank account? This is a common use case to pay for China-side costs and not have RMB exchanged to USD and exchanged back to RMB to pay partners/vendors.

3 - Will you have servers inside China? If you don't you will need some incredibly fault-tolerant javascript and page load magic as some of your interactions with non-China servers will fail.

jhancock··on Data Science Toolkit as Virtual Machine
Thanks! I'll give this a try. My data is in mongodb using ruby/rails as a web front. Data structure shouldn't be too hard...its just books, users, bookmarks, and tags/genre. All have unique ids (mongo UUIDs for users, books, and basic integer ids for the tags) and simple associations so I guess it couldn't be too hard to import. I'm thinking an offline system would be fine for some experiments. Not sure I need some real-time recommender integrated...but I'm new to this, so I guess I'm still not sure what I need.
jhancock··on Data Science Toolkit as Virtual Machine
I'm looking for an appliance solution for the textbook example of users-books-genres recommendations. I've read articles and books that present this as a classic problem but haven't seen a packaged appliance that helps me put data in and get recommendations out. Is there such a thing? Is there a reason such a thing can't exist without putting the developer through a big learning curve?

I have an ebook site with around 100k registered users and 1M bookmarks (what they read). Every book is well categorized with one or more tags. I'm guessing I have enough data to feed the recommender but have put off trying as the learning curve seems like more than I have time for.

jhancock··on All the Western companies you’d have to combine to get something like Alibaba
see my reply to 'dengnan. Alipay has users' trust and to them it isn't a terrible experience. Alipay is a bit different than Paypal in the way it integrates with your Taobao purchases. Alipay, at least for Taobao purchases, acts as an escrow agent and is or at least was strongly aligned with the buyer. Paypal has/had a history of generally siding with buyer grievances but the mechanisms of this trust is different enough that Alipay's solution is tailored to the problem effectively.

To your question "Is it the lack of competition..." I would say this is not the case. There has been plenty of competition. Alipay grew its adoption on its integration with Taobao in similar ways as PayPal was integrated with eBay.

jhancock··on The FuqIt Web Framework
I've been thinking along these lines for a while. For python-based, I think your toolkit is a good stake in the ground.

I think Lua-based is very interesting. openresty is a great choice for replacing the simple glory of what Apache/PHP once was.

Although, Lapis (moonscript/Lua) adopts "Web-MVC", the way its coded is a toolkit of un-opinionated libs. You can skip the MVC stuff and just have a clean set of libs for throwing things in a directory and getting your bang for very few bucks.

http://leafo.net/lapis/ http://moonscript.org/ http://openresty.org/

jhancock··on Riak 1.4 Released
Is there a Lua/openresty client on the horizon?
jhancock··on IBM Selectric Typewriter
I worked at IBM in 1987. My group's secretary was AMAZING! I remember walking up to her one day. She could talk to me in English, chat to her girlfriend on the phone in Spanish and type a hundred words a minute on the Selectric through her dictation headphone simultaneously and not miss a beat. Simply amazing!
jhancock··on Announcing Kiln Harmony: the Future of DVCS
If being fully git/hg interoperable was holding back sales, I can understand their year-long investment. What is holding back me buying into FogCreek's products is tightly integrated Trello-Kiln-FogBugz. Hope you get to this soon.

oh, and for the love of all of us doing dev in China, can some of these SaaS's please put a server over here so we can reliably use your products. Trello's single page app works pretty well from China. Most don't. I'm stuck with redmine and other things I can self-host.

jhancock··on Help Fight Patent Trolls – Support the SHIELD Act
Yes, it does seem reasonable to allow the defendant more choice on where they have to go to court.

Is there some hack around the East Texas problem? If you don't sell your product in that area, would it be possible to avoid being brought to that district's court? If so, I could imagine companies could simply have a notice on their checkout page: "Sorry, you live in East Texas, we don't sell there". I'm sure some companies would not want to leave money on the table like that, but for many things I do, I'd be happy to take such a position.

jhancock··on China And Open Source
Life sucks, yep. Low pay, yep. Management that has no idea how to run a business that creates intellectual property, yep.

Here's what you may not know: There are a growing number of small to mid sized IT shops in China that need good talent, know how to treat them right and are willing to pay better. FIND THEM and pick yourself up and things will get better. Go to these hacker meetups and learn to mix and mingle.

All my employees work only 40 hours a week, with benefits and pay well beyond the norm. And I invest in their future by preparing their skill set for what's to come next. I'm not the only person in China doing this. I know the man that wrote this article here is very progressive and invests in his people.

← PreviousPage 3 of 34Next →