Note: an updated version of this post with revised recommendations is available here.
As the agenda looked so interesting I decided to drive to Ullesthorpe (home of Combined Knowledge in the midlands) on July 22 2010 to the SharePoint User Group UK (#SUGUK). Considering I live in West Berkshire (and drive a 1993 Ford Fiesta, which I am rather fond of) this turned out to be a 5 hour round trip. Was it worth it? Hell yes! In addition to two very different - but fantastic - presentations from @mattgroves (big on social and lead me to believe that SharePoint is in fact an elephant) and @SteveSmithCK (capacity planning extraordinaire), I managed to win myself a 12-month MSDN subscription! Given that the question was basically "who has the rights to deploy a sandboxed solution in SharePoint 2010?", I consider myself very lucky to have won given the number of other SharePoint addicts sat in the room with me. I think the fact that most of the other attendees appeared to already have one (I heard someone behind me talking about how she planned to give her "spares" away) helped considerably.
However, whilst I enjoyed both presentations thoroughly, one question from the audience troubled me somewhat: "is there a limit to the number of application pools you can host on each WFE server in SharePoint 2010"? While Steve's answer was "around 100 before you start to run into IIS issues", I couldn't help but shake a nagging feeling that I had read the TechNet recommended maximum was 10. In fact, I was so damn sure of that TechNet number (10 app pools / WFE) that I would have bet my brand new MSDN subscription on it.
Now I was faced with a dilemma here: did I take Steve Smith's guidance as gospel, or go with Microsoft's recommendation? I did what I figured every sensible SharePoint administrator would do in my situation: I sat down, got back in my box and reminded myself that on the odd occasion TechNet tells more fibs than my parents used to on Christmas eve.
I did however make a point of looking up the TechNet article in question this evening. It does indeed list 10 application pools per Web server as a recommended guideline - in this case the "Limit type" column indicates the figure is a supportability number. Now, I know MS supportability numbers and they are not typically less than you would expect - and I would certainly not expect it to be ten times less than the guidance provided by a SharePoint legend such as Steve Smith. As a recent example, @harbars kindly informed me that having over 50 Web applications in one MOSS farm was, in his words (tweet) "madness" - Microsoft recommend no more than 99 Web applications. I think you know which guidance I paid attention to and he doesn't live in Redmond.
To be fair to Microsoft, the article above does also state that the maximum number is largely determined by hardware capabilities and the workload that the farm is serving. Steve also confirmed this at the user group by telling me that the app pool ceiling has been lifted dramatically due to the move to a 64-bit-only architecture for SharePoint 2010. Now I know that 64-bit brings about numerous performance and scalability benefits (not least of which is a practically unlimited, continuous address space for user mode processes), but I am still keen to find out why there is such a huge discrepancy between Microsoft's stated supportability figure and the guidance of industry experts who have used the product for the best part of 2 years.
So, rather than pester Steve again, I did what I often do when I have a nagging capacity-related query: I sent the query to a few friendly folk in the SharePoint community who (despite the fact that they are quite clearly very, very busy people) normally find the time to respond to my numerous concerns. Did anyone reply this time? You bet.
@ToddKlindt gave me a pretty clear response by saying that "anything Steve says, I believe". Sounds like good advice right? He also told me that the documented MS advice was actually pretty good considering that they suggest the limitation is largely hardware dependant. For those of you that don't know Todd, he is a well respected SharePoint guru and a co-author for Professional SharePoint 2010 Administration. If you don't have it already and you are a SharePoint 2010 administrator I can whole-heartedly recommend that you add this to your one to your collection of geeky paperbacks.
@joeloleson's article entitled "What’s New in SharePoint 2010 Capacity Planning" also mirrors the advice kindly provided by Steve and Todd. Joel points out that Microsoft have adjusted some of their capacity planning guidance based simply on the fact that IIS and SQL scale better with 64-bit, and states that the number of application pools " totally depends on the RAM on the box".
So how many Web applications can you host on a SharePoint 2010 WFE server? It looks like the usual SharePoint consultancy response: "it depends on your hardware's capability and server farm load", and there is no magic number other than the guidance provided by Steve that you may run into IIS issues as you approach 100 pools on one server. Microsoft's supportability number of 10 is there as a rough guide (probably for those running a minimal WFE hardware configuration - think 8GB of RAM), but is almost completely dependent on your available hardware. The new SharePoint 2010 Administration Toolkit includes a load testing kit that should help administrators validate their hardware requirements as part of a capacity planning exercise.
I hope that clears things up a little bit for anyone else concerned that they have more than 10 application pools on their SharePoint 2010 WFE servers. I'll see you at the next #SUGUK!
Just to add to this - I did confuse the web apps limit with the app pools but just to extend the topic here a little, The Conversation was very much geared to the capacity handling of IIS in Server 2008 R2 and 64 bit. The Technet article is very specific in that the supported limit of 10 App pools is also subject to the machines hardware and what is running on that server. If you had 5 App pools all busy then you could easily consume 24GB+ of memory and all four cores very easily on a 64bit WFE Server. It is important to note that in IIS 7.5 you can control almost every expect of how many resources a worker process can consume (In fact you can specify the maximum number of worker processes as well) and although the current guidelines by Technet still state 10 for SharePoint IIS 7.5 can handle a lot more if managed properly. Does that mean you can go off creating dozens of applications pools on one server for SharePoint, absolutely not the reality is before you reach more than 6 let alone 10 you have already reached a point where you need to scale your servers and have dedicated applications servers that have app pools dedicated to that service and in some cases only 1 service would run on a specific server such as performance point or Excel. So the other questions then arises. How does a company that has say 3000 people support all those people and all the required services on a single server ? According to Microsoft's planning guides one server with the right hardware should be able to handle that + 2,000 more but obviously this will mean that some of the web apps / services need to share app pools. It also depends on what type of users the 3,000 are and how many are concurrent and what services are running but either way 1 server is not an ideal situation when trying to host several very busy services and several user hosting web applications also busy. So the only way forward here is to create more application pools in order to isolate and control services and content web apps. Well as we already know each app pool will end up consuming far too much on one server and so multiple servers now becomes the only logical way forward. So in summary can IIS 7 handle more than 10 application pools - of course it can and the more you create the more you need to manage them and define resource allocation. But can SharePoint make use of so many on one server the answer is no because the reality is before 10 you will already start to run into performance issues on the server.
ReplyDeleteHope that helps to clarify a few of the poinst we were discussing
Steve Smith
We are in the process of installing MOSS 2007 in a 64-bit windows 2008 server. Our current environments are 32-bit. The app pool settings for the 32 bit servers set “maximum worker processes” to 1. Should we keep this at 1 for the 64-bit server?
ReplyDeleteThe "maximum worker processes" option should always be set to 1 for a SharePoint application pool.
ReplyDeleteAFAIK Web Gardens (i.e. more than 1 worker process) are NOT supported with SharePoint.