Author here, this is using free Microsoft tools only. Was a big hit in our troubleshooting course, so we figured we release it for public benefit.
51 karma · joined October 16, 2012
The first thing I would check is that you can hit the webserver. You can do some of that with a tool like PortCheck: http://mvolo.com/check-iis-network-connectivity-with-portche....
Best, Mike
People on HN don't often talk about it, but it does happen, and apparently quite a bit judging by our users :)
That's certainly a good one! One of the things we'll do to help resolve the hang is to identify where blocking is happening. This will pinpoint the session state module in AcquireRequestState as the culprit.
Although, I also have to say, IReadOnlySessionState wont help you a whole lot in a high volume site because the SessionStateModule will still connect to SQL server to keep the session alive in ReleaseRequestState. So, best advice I would give is to use the new AppFabric Cache based session state or go stateless.
Best, Mike
But the bottom line is, even if not everyone likes you in the US, you can still live a fully actualized life even if it takes a bit of hard work. In Russia, institutional racism created a hard ceiling that most of the time could not be broken, even with hard work. If you tried to hard, you would go to jail or worse.
The post is about selecting a top level metric for monitoring website performance. One a problem is indicated, you would definitely need to drill in to figure out what part of your app is affected, when, and what caused it.
LeanSentry (our own application monitoring product, https://www.leansentry.com) does this. However, describing this was outside the scope of my post (but you can see the demo of it on the website).
Author here. Did you also search for stdev, st.dev, variance? Just kidding.
The post is not about averages. Its about selecting the right metric to track website performance. Standard deviation would surely qualify the avg. latency a bit, but it would still be a pretty lame alternative to using a better toplevel metric like Apdex.
Best, Mike
Unfortunately, you HAVE TO do it. If you do not set a threshold for what is acceptable, how do you determine whether or not your are providing an acceptable experience to your users?
No amount of aggregate metrics can help you answer this question unless you know whats acceptable, and what isnt - for each important set of URLs in your app.
I agree that its "hard" to do. In our own product (https://www.leansentry.com), we solve this problem by grouping urls, and using good defaults / making it easy to override the thresholds for users.
Author here. I think most people on HN would echo your sentinment about averages wholesale ... But I wanted to go a little deeper into selecting a better alternative for operational monitoring.
Its easy to say "averages are bad" but harder to say "use X instead", and explain why. We tried. Do you think we did it?
I completely agree! Keeping toplevel metrics SIMPLE is the key. Of course, simple but also not misleading you into any wrong beliefs.
While we liked the 95 percentile approach, we decided against it. Its still too focused on the actual response time itself, which we thought was less relevant than the number of users experiencing bad performance.
I think for us the bottom line was:
A) If you are having a site-wide performance issues, 95% percentile is a good metric.
B) However, if you have more isolated issues (we find this happens more often to more mature sites), satisfaction score is better.
Best, Mike
Author here. The 99 or 95 percentile is a much better metric! We also make the case for the industry standard Apdex or our derived metric, sat score. These are becoming more and more in use by APM tools like us or New Relic.
Unfortunately, many existing tools and people who use them still look at latency aggregates and often make incorrect assumptions.
I am in complete agreement. Unfortunately, a lot of monitoring and APM tools still lead with average response time as one of the toplevel metrics. And a lot of people still make incorrect assumptions based on it.
Although, the percentile on average latency is not great either. I try to make the case for using a metric that counts acceptable experiences vs. their latency value, e.g. the Apdex index or our derived sat score.
Best, Mike
1. You are absolutely right, the agent is an optional component. Without the agent, you get most of LeanSentry's monitoring features without having to install ANYTHING on your production servers.
2. We know the name of the stored procedure/SQL query from monitoring ADO.NET activity externally via ETW, the high speed event tracing in windows. We do not actually connect to your SQL server for this.
3. We worked hard to make sure LeanSentry is safe for production environments (thats what its for). We also designed it so it can be completely sandboxed in production. Most importantly, it will never load anything into your application directly, unlike debuggers or profilers, and has no chance of crashing your application.
The agent adds some deeper diagnostic capabilities, including SQL query/IIS module/REST call monitoring, CPU profiling, etc. It does have a bit more overhead since you have to install it on the actual production server, but we worked hard to make sure it keeps a low footprint (and even shuts itself down if something goes wrong causing higher utilization).
4. You can remove the monitoring service and the agent anytime. We provide the configuration script for servers as a batch file so its easy to see the changes we make.
These are great questions - if you have more, hit us up on chat on our website!
Best, Mike
We are making the pricing page more accessible, should be live later tonight.
Re: the pricing for 30 servers.
1. If you use our Core plan, which gives you all our features except the automatic troubleshooters and expert insights, you will pay about $610/month on an annual plan.
2. If you use the Advanced plan, that means you are finding enough value in us automatically resolving your problems and giving you proactive guidance. On that plan, you are paying 2295/month on an annual plan.
I know this might sounds like a lot, but its actually reasonably lower than some of our cheaper APM competitors. The bottom line though is, if you are professionally running a web application, this is chump change compared to the time this saves you:
- If you are a 1 person shop with 5 servers lets say, you may be paying ~150/month. If we just saved you 3 hours this month, you paid for it already.
- If you are a company with 30 servers, you probably have a lot more at stake in making sure your servers run well. But also, you probably employ several full time developers and ops people who manage your servers. You get my point I am sure.
This all depends on whether we are able to truly generate huge time savings in your work day by eliminating the time consuming investigations and troubleshooting. And thats exactly what we are shooting for!
We also have a promotion right now with a 20% off rate, free upgrade to our Advanced plan for 2 months, and great prepay and volume discounts. More pricing information here: https://www.leansentry.com/Pricing.
Best, Mike
I hear your sentiment re: hosting LeanSentry yourself. We actually started this way, but over time realized that in order to support our data processing and analysis needs, we needed to host this in our own data center. The majority of our customers didnt think twice - they just install it and done.
This model has been proven by New Relic as well - just look at their growth numbers over the last few years.
Certainly there will be people that are more concerned about security, and we get a fair share of them talking to us. The best we can say is, we've done everything we can to make LeanSentry safe (see my earlier comment) while still preserving the value we provide.
Best, Mike
You are right, most APM tools are working on covering multiple platforms. We are trying to provide extra depth into the Windows Server platform because thats what we know (for example, I used to be the PM for IIS 7 / ASP.NET).
Our goal is to be able to help you with more than just giving you the info to monitor your apps, but really give you the expert analysis of it. A better example is, you might hire a consultant to help you tune up your app, or have your developers get training on performance tuning. Those developers will then potentially use New Relic or Redgate tools to support them in doing their work.
In our world, you just use our product and receive that level of expertise automatically from our service. Without having to hire those consultants or have your engineers spend their time learning about the insides of IIS.
Re: leaking information. If you are not comfortable with leaking information, you are not using the new SAAS APM tools like us or New Relic. Thats certainly your tradeoff to make. I have to say though, we worked really hard to make sure we dont collect sensitive data:
- We dont load a profiler/debugger into your application like most other APMs (e.g. New Relic). We collect only external information, which does include errors/stack traces but no data/code/cookies/etc.
- We provide a mode where our monitoring service is sandboxed to a non-production vm in your environment, so nothing to install on prod servers.
Re: scale. We've handled Viddy.com, Vevo.com, etc. Are you larger? We do a lot of engineering to reduce the data volume and process it effectively while maintaining high resolution. Tell us what your specs are and we can tell you if we can do it.
Best, Mike
LeanSentry is somewhat similar to New Relic in that we provide server and application performance monitoring.
However, we are 100% focused on the Windows Server/IIS/ASP.NET stack (that's our background), and as a result provide deeper analysis and diagnostics for those applications.
Best, Mike