Recreation.gov
recreation.gov
recreation.gov
One of my coworkers wrote a bot and shared it around the company. Not sure how far that bot spread but I’m sure other people did the same.
And by bot I mean a pretty trivial selenium script. The parks really should have had better defenses.
Requiring a deposit payment at time of booking is a pretty easy one. Make it non refundable if it continues to be a problem. Increase the price if it still is a problem.
If the goal then is to make it fair for everyone regardless of ability to afford it, require a legible scan of ID or passport and restrict each person to reserving 1 spot in the future at a time, and they must be the person stating in the camp site so someone has to verify their ID at the camp site too, otherwise they can now resell that reservation like ticket re-sellers.
I don’t see other options of “fairly” distributing something with limited quantities. Trying to do it on the technical side by trying to prevent automated scripts is a losing battle in the long run.
The alternative of manually checking every day would have likely resulted in never getting a spot. I was using my skills to hack together a solution for myself in a crappy situation.
While I don't think my approach was too harsh (I wasn't constantly querying nor booking automatically), it could be solved if they provided their own alert system, or discourage the book-cancel behaviour (higher cancel fees? Might not make much sense since the camping itself is pretty cheap...)
https://www.fs.usda.gov/Internet/FSE_DOCUMENTS/stelprdb53637...
State parks are terrible from the fees, crowds, and rules. It’s more like camping in a grass parking lot than anything else.
It’s much closer than my other spots. I’ll check it out soon, maybe even Friday. Looks like they just stocked Amicola creek with trout.
Go slow, keep your eyes open, and you'll probably see a spot which looks perfect to camp at in no time.
Lands managed by the BLM or USFS usually allow dispersed camping in at least some areas, but it's still a good idea to double-check before you leave, and you should be aware that the same lands are often used for other activities like hunting, logging, horse-riding, grazing, offroad vehicles, etc.
You can also get more specialized outdoors mapping apps which can download data like USGS topographical maps and quad sheets for offline use. If you think you might like the backcountry, a satellite messenger is a good investment and those often have companion phone apps.
Don't forget to bring a big battery if you rely on electronics for mapping, though. And you can usually get decent paper maps from local NPS/USFS/etc visitor centers once you get to the area. The people in those visitor centers will also tell you where good sites / hikes / drives / views are, but then you have to actually talk to someone.
Also, read about pack-in/pack-out policies, what a cathole is, and how to keep your food from attracting animals.
Long ago, I ran part of the backend systems for a contractor that had a variety of state and other parks. One set of islands on a lake in particular were very important to a number of people for various reasons.
Specific weekends were in particularly high demand, and people discovered gaps in the booking and rescheduling policies that would allow them to reserve the spot at a different time, then do multiple re-books. IIRC, they would spend about $500 in re-booking fees and would have to coordinate re-booking in multiple specific, inconvenient time windows. (Part of it involved a window where the phone system would accept changes before the online system)
Somebody figured it out, and it escalated to the point where the legislature was investigating it and one of the VPs had to testify to a committee.
Edit: This seems to explain it - https://news.ycombinator.com/item?id=21625871
My thoughts exactly.
Online camping spot reservations need to ask for your name at the time of reservation, and ensure the name matches your ID when you arrive at the park. This is how many (non-Ticketmaster) concert e-tickets work.
(ticket raffle not up yet for 2020, though)
is there? whats the source.
...you are joking, right?
Further, it's the start of hunting season in a lot of places, which means lots of people are out hiking and camping to hunt.
Looks very different from a few years ago.
Brief Shoutout for anyone who would like to see a better online experience for our National Parks System:
1. write into rec.gov asking for a booking API!
2. join accessland.org
3. build apps using Rec.gov's availability API
(disclaimer: I'm involved with accessland.org)
The Federal Parks system took the first step in creating an availability API (with an intention to open up bookings via API as well). California then forked the language (with minor revisions), so they are slated to have an Availability API "soon" (who knows when it will actually come). This sets precedent at the federal and state level for parks, and also establishes a data schema that all the other state, county, city, local govt parks can build into, which would allow us to have a fully integrated discovery system for the public when looking for places to get outside.
A big part of persuading governments to follow is showing examples of what's possible, and the Rec.gov information & availability API, however limited, enables 3rd parties to do just that.
An advantage of an API is that keys can be revoked for bad actors, so it at least gives some measure of control where there was none before.
I tend to agree that the lottery is the way to go for popular sites instead of first-to-book; it's fair and stops older folks from never getting a booking because they don't know how to build a faster bot.
There are a few other very effective hacks that I am loathe to publish here, but there _are_ ways to get sites for the determined.
https://www.nps.gov/findapark/index.htm
https://www.blm.gov/programs/recreation/
As for campsite reservations, peak seasons will always be terribly crowded at major campsites. It's too bad that people are gaming the system, but if you need to rush to get a spot in line, you're probably not going to feel like you're out in the wilderness when you get there anyways. So if you aren't a huge fan of crowds and noise, try going during the off-season (like right now!) Also, consider checking out smaller state forests and parks during the summer months. And many forests and parks allow 'dispersed camping' in certain areas; you can usually find plenty of semi-maintained campsites along roads in those places when the maintained campgrounds are full or closed.
And if you go to these areas regularly, an annual parks pass might save you money, even if they usually only get you discounts on federal land:
In the end our favourite spot was a quiet campsite in Lassen Volcanic park.
https://www.wsj.com/articles/seeking-happy-campers-governmen...
Edit: the National Park Service also has its own API: https://www.nps.gov/subjects/developer/api-documentation.htm
(If you can't find a signup form on the rec.gov website just contact them directly and ask for access. I think their team was rolling access out slowly as they hadn't finished building a fully fledged login system but last I checked they do want 3rd parties being able to build on top of that data)
Booking is not.
Note: If anyone has examples of startups accepting payments on behalf of the government please include them here. It helps with research. (Tax filing companies are the ones that come to mind... looking for other examples.)
They disrupt soil, more often than not made out of live cut trees (and hinder wood decomposing when not) and 9 out of 10 times, will come with beer cans/tp scattered around and it's illegal to 'build' them on public land.
If you happen to see one, destroy it, you'll be doing well for the environment. In fact, USFS and NPS often do just that as part of their job.
Here is one guy who's work on that field inspires others: https://douchepacker.com/2018/07/09/destroying-buschcrafter-...
My guess it's downvoted because maybe some readers here are offended that they're being called out.
Anything to feel superior I guess.
Not sure if it was you asking about it that got it upvoted or the additional explanation. I'd guess both and I'd further estimate that most people that are not involved in the hiking/preservation community could pass near one of these 'shelters', without recognizing that it's bad for the environment. It's easier to recognize a call to destruct structure such as this as a vandalism.
It is illegl to build a structure, bury food, etc on public land. You wanna be a prepper, buy your own damn land. Stop destroying ours.
P.s. kick over some carins too.
Be careful with that, though. Some times they’re used as trail markers.
Last year I was on a trail in the Southwest that had loads of cairns, like every 20-30 feet. Rather than feeling disgust at silly Instagram tourists that must’ve built them, I made a game of kicking over as many as I could.
When I returned to the trailhead I saw a sign asking to please not touch the rock stacks because they mark the trail.
Oops.
That felt awful. On the drive home I made a donation to the park so I could go on living.
The original Steve Roper's guide book for its brainchild — Sierra High Route, opens up by instructing hikers to do just that so it doesn't end up becoming a well worn path. It's probably not a good test case for whether or not it's good for the environment since very few people hike this route. Anyhow, the vast majority of (official) trails in the US are so well marked and maintained compared to any other country I've visited, that cairns are nearly always redundant.
Additionally, in an extreme environment like that a single well-worn path is far less impactful to the environment than wandering all over in generally the right direction.
I totally agree that cairns are a good marker when there is an official trail that in parts has no visible trail (usually in boulders/lava/talus fields).
btw, which desert of the southwest is in question? That's a LARGE area. If we're talking SoCal for example, the entire 700 first miles of the PCT are desert with trail so well maintained and visible that when I hiked it last year, I walked many nights well into night without having to take my flash light out until it became pitch black.
I could imagine there are trails in southern Utah and northern Arizona that might be heavily cairned
Yes, I can imagine that being the case, I'm actually reading Desert Solitaire by Edward Abbey, mostly about the times he worked in Archs National Park and it sounds just like that.
Fun fact you might share - I stumbled upon Edward Abbey following a quote I saw on a blaze in NorCal (which carried bullet hole, obviously, but it actually what made this quote stand out to me): https://pbs.twimg.com/media/D5bbY00X4AA_eeu.jpg:large
It was misattributed I think, I later found out these were Abbey's words.
I'm convinced the only reason there's not MORE scripts around to do it is because their API/web framework is impossibly ugly and not intuitive.
Anyone botting the site should feel ashamed.
I just wish there was a "calmer" way. The experience of exploring our national parks should be an escape from our hectic every day lives. That begins with getting a place to sleep.
What I was pointing out is that a lottery does not on its own help with bots. Bots are good at making reservations very quickly at a specific time of day (in reservation systems that are first-come-first-serve), but bots can also easily be scaled up in number to sign up for an arbitrarily large number of slots in a lottery.
> The experience of exploring our national parks should be an escape from our hectic every day lives.
I totally agree, and it bums me out too. I have visited Yosemite several times per year for the last 6 or so years, and it's only gotten more and more popular. I rarely visit the valley any more, because it's such a madhouse. Even in the last few years I've seen demand go through the roof even for relatively difficult wildnerness backpacking trails. But the fact of the matter remains: way more people want to go than there is capacity for, and I'm sure they all want an easier calmer way to get reservations.
Imagine if campsites were either: a) first-come first-served b) reservation by physical mail only
I guarantee you'd see different levels of "demand" than we do now.
Interestingly I noticed that Yosemite backcountry permits still require mail or fax, but they email you the result. I think this is intentional, putting a small hoop to make sure they get "serious" inquiries only.
I say all of this out of defeat. On principle I would want to remove any and all barriers to visiting our national parks. But in practice what we have is not really working for anyone (at Yosemite and the other mega parks)
> But in practice what we have is not really working for anyone (at Yosemite and the other mega parks)
I'm not sure that's true. Tons and tons of people go there. It's not like it's so crowded that no one goes there any more (like the classic joke).
I use recreation.gov and the WA state equivalent regularly and am able to get decent reservations pretty easily. During the really high use times in WA we just go to camp sites (Forest Service sites are particularly good for this out here) and see if they have walk up spots available -- all campsites we have visited keep at least some spots available for this. Granted, we have only done this maybe four of five times so far, but we have got a spot every time.
They also seem to have a nice collection of KB articles under the 'Help' link [0] in the header.
To add insult to injury, the landing page has a 2118x1415, 2.6MiB background image.[1] Doing a `-quality 70 -resize 50%` conversion to JPEG results in a mere 176KiB.
Although, having experienced service-now.com in other settings, it definitely could be worse.
[1] https://recreationonestopprod.service-now.com/83f45a56dbbbab...
Sites that don't have electric or water hookups may be easier to get since they're less popular with RVs. And the folks with RVs are experts at campsite reservation.
That said, go off the beaten path. Less people make for a more enjoyable experience. Some of my best car camping experiences have been on the side of a forest service road, especially one accessible only with a high clearance 4x4.
Sites that are accessible only by foot, even a half a mile in, are usually not that crowded. But you'd have to backpack in everything you'd need for the stay.
I notice the app is made by Booz Allen Hamilton. It's a shame we have to farm this stuff out to contractors and not use the USDS or 18F.
Thanks, fixed.
And to your original comment. I'd also love to see the government building out solid technical teams for provision of services. I'm also a firm believer that for _some_ government functions, there's a strong argument for "government as a platform". Rec.gov is a great test bed for this; among many reasons it's (mostly) politically impartial, so is less.
For at least the online portion of parks reservation, we really want to be offering availability and booking via API, and incentivizing 3rd parties to compete on experience (UI/UX, etc.).
It gets tricky because this is newer terrain and areas like "who pays for support costs/requests" is still being hashed out.
Note: still important to have the government providing the base layer of functionality. Accessibility is important and capitalism won't solve for every piece of product minutia that government should provide.
Duration of the performance-based contract will be three and a half years, with a total potential value of $128 million. It provides for additional options of up to 10 years.
https://gcn.com/Articles/2004/08/09/USDA-awards-contract-for...
Sooooo yeah.
We can pay people to build it directly, it's ok. There doesn't need to be an unaccountable greased palm making $100 million in the background through some "public-private partnership".
If the contractor makes a profit we pay more. If instead the government makes a profit, we pay less (see public banking for an example). Kinda funny how that works.
Norway uses one for their pension program (AUM of > $1 trillion, owning over 1.4% of the global stock market, https://en.wikipedia.org/wiki/Government_Pension_Fund_of_Nor...)
My favorite one, however, was created by Ronald Reagan, the United States Mint Commemorative Coin Program:
> Since the modern commemorative coin program began in 1982, the United States Mint has raised more than $506,301,189 in surcharges to help build new museums, maintain national monuments... and much more.
https://www.usmint.gov/news/inside-the-mint/commemorative-co...
Honestly a world where the government has, say a commercial for-profit version of NPR, high speed internet, etc, that coexists with other current offerings and then re-invests that into government programs and needs less tax dollars from me sounds like a great idea. Tax payer money already paid for most of the infrastructure, we might as well profit from it.
There's some odd political animosity to it however, as if asking for a tax-payer owned version of NBC with some entertaining primetime shows is like some kind of slippery slope to autocratic socialism ... it's actually bringing capitalism into the government like never before and allowing them to say, not have to ask me to vote on raising my sales tax to build a train. Instead they ask us how we want to use all those sweet profits.
... it'd be nice ...
My takeaway from watching this whole RFP process play out is that an early impactful leverage point is having USDS get involved in auditing/writing RFP requirements in the public interest (this was quite a few years ago, so they may already be more involved in all tech contracting decisions).
Without having a technically strong government team weighing in on contracting, I get the feeling that language and technical expertise falls on contractors/corporations/lobbyists to provide.
I worked on a project once with the national park service. Their Office of General Council or whatever its called didn't want to agree to make their metadata on https://npgallery.nps.gov available to use with a CC0 license for use in https://dp.la/ . Sure, the images might not have been taken by federal employees, but why try to assert copyright ownership in the metadata? Eventually they agreed for one collection, but they have lots of cool content on there from California parks we would like to put on https://calisphere.org .
The stuff they have up at Clemson has CC0 metadata, and the rights info for the content is pretty clearly marked https://openparksnetwork.org/about/opn-rights/
Mine worked. Lovely site, knew exactly what I was headed to, and got my permits. Seems others had issues if they had to search around. Entire thing sounds like it shook apart later that morning. They canceled the reservations and did a second 'launch', which was the right thing to do. I don't know of anyone who thought they had booked, canceled, and could not get a permit later. Successfully turned a terrible kickoff into something OK.
Also the Netherlands memorial is located in Belgium, but in the "How to get there" instructions it is said to be east of Maastricht.
Disclosure: I don't work at Booze but did collaborate with them loosely when integrating rec.gov's API[0] into Hipcamp.com
The previous contractor was Reserve America. If you're interested in some fun reading on open data advocacy that led to everybody being able to access public parks data via API check out accessland.org
It's not really all that hard to understand, it appears on the face of it to be supply and demand. There are a lot of people throughout America who love camping and not all that many sites at the popular locations relative to that demand. Has been this way for many years. I imagine the problem is likely particularly acute in places such as the Bay Area given the population density and climate, as well as the more popular National Parks like Yosemite, Yellowstone etc. If you haven't recently visited one of the major NPs at peak season you might be genuinely surprised how many people cram into these places.
I've taken recently to just booking whatever I can get - I often don't have much choice but to take what I can find that's actually available.
Even if it's a "global rate limit" I'll find out the value (never ran into someone randomizing it) and jump on the web request faster than anyone else RIGHT as it comes up.
With CAPTCHAs I'll bypass with a solving service and/or computer vision if it's easy, or even just get past the noCAPTCHA solutions with primed browser instances from credible networks.
But don't kid yourself - that would not solve botting at all.
What would solve it? Or rather, what is the best defensive measure these days?
Even the reputation-based stuff is laughable and one can hide a Puppeteer instance with good originating networks, and spoofing a ton of details in the browser. Even if that's a no-go you can also automate plain-old-Chromium/Chrome with extensions and run it in a headless session through something like Xpra. I'm experimenting with Firefox solutions as well.
All-in-all, I've never been stopped - and that's not me stroking my ego... there's TONS of resources out there for this stuff that are just a DuckDuckGo search away.
The biggest thing is if they start aggressively fingerprinting bots, they're going to start blocking user real people. It's all based on a score - and getting a good score is just a matter of a credible proxy, CAPTCHA bypassing services, and making a browser look highly credible.
---
For a "real" answer of some value - as a web developer myself, I'd try to make it as expensive as possible for them. Which specifically would be to implement a non-standard CAPTCHA solution and do rate/conversion-limiting per-network. The reason I didn't say this up-front is because it's not a solid solution - it's just increasing the barrier of difficulty and cost for those that are trying to automate around your solution.
It's a cat and mouse game for sure, but the answer is usually nothing if the person is sophisticated enough. You can only increase the difficulty/cost (covered in that comment).
Surely capatcha's and rate limiting raises the bar for people botting.
It couldn't make it any worse right?
For something that's highly desirable like tickets, Nike drops, or apparently campsites there are many people with sufficient ability to bypass this stuff.
To paraphrase Sinclair: It is difficult to get a man to divulge how to prevent something, when his salary depends on his not preventing it.
Refusing to give away a secret has absolutely no overlap with Sinclair’s saying.
I thought "paraphrase" was sufficient, but I appear to have misjudged.
Here's the thing - all of this info is out there (largely in other HN threads on this topic) and I'm nothing special in my field-of-knowledge ;)
I'm confident if people fully prevented my "abuse" they'd start to block actual users... simple as that.
When it comes to a web "interaction" there's a ton of differences depending on the job at-hand. If it's jumping on a Nike drop you're not going to run into those sort of methods where-as if it's scraping hundreds of thousands of SKUs there's a huge chance you will.
When a human opens a "conversation" with a website usually it's for a specific action which is a small burst of data, ie: looking at a few products on Amazon. So as long as you can design your bot's "conversation" with said website in the same way you can totally bypass ML detection (and everything else for that matter).
Basically if you think of ALL of the unique variables that are happening back and forth in that conversation it's a finite list. If you can check off each and every one of those you can get past anti-bot measures no-problemo, it's just a matter of budget/time. ML can HELP to identify bot traffic, but if you're sending perfect headers/traffic/browser metrics/CC numbers/shipping addresses (this list gets long) you're still going to squeak by with an acceptable risk score without a problem. The other thing is going raw-HTTP request often bypasses almost all of that crap, way more than one would think! I will often get "fingerprinted" with a very valid browser, then continue my work after treating their site as an API; ie: talking directly to the backend (or rendered HTML page) and no one else.
The #1 thing I'd say where bot classification works is network reputation... so I use services that allow me to proxy-out to VERY reputable networks that are actually business/residential ISP connections. This lets me get past the majority of countermeasures because if they start blocking those sort of IP addresses/ranges they're going to block real users at some point. Unfortunately, highly reputable proxy services do cost money - but for someone who does this professionally it's all about budgeting etc.
To implement these systems well takes a really keen implementation between your front-end and back-end and most companies (even the big ones) don't have the development sophistication to pull it off. Those that do are just more expensive because you'll be bouncing around between 40 different "origins" with unique browser profiles.
Sorry for the essay =)
This seems like just the kind of behavior that an ML approach could easily identify. Even just feeding a trained model your basic request log data would show you as quite a different kind of user: you're not fetching images, javascript, etc and would have a substantially different traffic profile. Obviously you could get around that by scripting a browser, but that just kicks the can down the road: a scripted browser will still likely behave in some measurable way different than a human-driven browser, and the specifics of those differences are unlikely to be found by intuition but rather by ML which can hone in on things we wouldn't think of. For example, the time spent in certain pages/activities, or the position of the cursor on links being clicked (when you tell a headless browser to click a link, do the mouse coordinates in the click event look normal or are they at the upper left coordinate of the link's position?) etc. The more data you feed such an approach the more details it can use to find anomalies that differentiate humans from bots.
Any recommendations for books/other information sources? Currently doing some backend work for a company that mainly does scraping and a lot of this seems to be based on the tribal knowledge of the resident old wise one.
Stuff like https://news.ycombinator.com/item?id=20479015 is a goldmine for me and it usually is a fun weekend working around said methods in a lab-like environment.
> resident old wise one
He's just invested the time in picking this skillset up - it's definitely not just something you're an expert on after a few small projects. It takes years of having things break over, and over, and over [...]
There's a small Discord server that I setup for people who do a lot of RPA/web scraping if you're interested in joining a "tribe". My contact info is accessible through my profile if you're interested =)
If I were building a bot for a site that uses captchas, I'd just use one of the many paid services that solve captchas. Last I checked, it was ~$0.003 per solve, hardly a blocker. To be honest, I wish I could easily use one of these solvers in my normal web browser...
His sign up page was all sarcastic and the recaptcha test asked
{ Image of distorted random letters and numbers } "Are you human?" [text box] Prove it...
And I kept trying answers like.
"Okay blood's drawn, where do i send the sample?"
"uhhh so I'm blind..." (no voice alt test)
"a robot testing a human for humanity? (this is going to go nowhere good quick...)
"Morty, its your grandpa Rick! Let me in Morty. belch"
"I identify as a pro exclusionary Cylon. My prefered pronouns are su, pid 1, root, or if you prefer just refer to me as Zuck" (okay i didn't get to enter that last one in. too long)
"Isn't putting random numbers and letters the kind of answer a ROBOT would make. It's only an answer written in THEIR language! o.O
Unfortunately Mr. Harmon's test had a fatal bug or something because I never could get past the sign up page. :'(
Or maybe I'm not human or i'm just missing something? idk.