Dear followers,
Recent related blogs that might interest you:
16/02/2010 Update: Having recently been to a SP2010 admin course with Combined Knowledge, I took the opportunity to discuss this with @SteveSmithCK who has suggested that for now we should ensure farms are in a supported state and stick to the MS documented limit of 10 app pools in total (i.e. Portal content AND Service app end points). I will update this post if Microsoft do change the SP2010 software boundaries Technet article to clarify this point. I would say that the article is currently ambiguous and open to interpretation - especially given that it states that "The maximum number is determined by hardware capabilities". Only the Microsoft product team can really answer this question once and for all.
22/12/2010 Update: Note that this blog post refers to application pools used for hosting content Web applications only, NOT those hosting service application end-points. This article by @jremmc explores the differences between the two - check it out!
A couple of weeks ago I blogged about the application pool limitations in SharePoint 2010 based on a discussion I had with @SteveSmithCK at the #SUGUK. Since then, I have been on vacation in Iceland and received a number of related comments from members the SharePoint community. You can read the original post here; in this post I will aim to provide a concise summary with some revised best practise recommendations.
The key points, which were kindly sent to me by @SteveSmithCK and @harbars are that:
Both Steve and Spencer made a point of emphasising that performance issues will likely be encountered even before 10 application pools is hit: "If you had 5 App pools all busy then you could easily consume 24GB+ of memory and all four cores very easily on a 64bit WFE Server". So although 10 is the documented figure, it is neither a "magic number" nor a hard limit imposed by the software and depends largely on farm usage (think user requests and service applications, particularly resource hogs such as Performance Point).
Although probably a topic for a future blog post, I think the fourth point above (from Spencer) is particularly interesting. TechNet document a number of reasons why administrators may wish to create additional IIS 7 application pools including grouping similarly configured sites and isolating those with unique configuration settings or security requirements. In the context of SharePoint, the primary reasons I have come across in the past for additional application pools are:
I think that there is some validity in the first point for clients that have very stringent security requirements and expect their data to be completely isolated - we have dealt with clients in the past that have requested separate pool identities as part of their corporate security review. However, one could argue that the new multi-tenancy features in SharePoint 2010 (site subscriptions, service application partitioning, host-named site collections; more over at harber.net) provide a solution to this problem and should help keep the number of required Web applications to a minimum. Remember that more Web applications always means more timer jobs and higher hardware demands - particularly if using a separate IIS application pool and associated worker process.
I think there will always be a place for an isolated application pool for troublesome sites and applications. Although numerous features in SharePoint 2010 should lead to higher availability (e.g. sandboxed solutions, list and HTTP throttling), I am sure that there will still be occasions where farm-wide custom solutions cause issues (e.g. a problematic Web part deployed to the GAC).
Another reason that IIS sites are typically placed in a separate application pool is differing .NET version requirements. This wasn't really relevant to MOSS given that versions 2.0 through 3.5 used the same version of the CLR (see this post for more information), but could be relevant to SharePoint 2010 going forward if Microsoft release an update that supports ASP.NET 4.0.
To summarise, the TechNet recommended maximum of 10 application pools is correct and great care should be taken in planning the creation of new application pools and corresponding Web applications. SharePoint portal application pools can eat a considerable amount of RAM and can bring down even the meanest Web server if deployed without proper planning. As a best practise, the number of pools and Web applications should be kept to a minimum in order to increase farm performance - I think it's worth remembering that site collections are the unit of scale in SharePoint (2000 per content DB recommended maximum) and the new multi-tenancy features in the 2010 version of the product should be utilised in order to deploy an appropriate (scalable) logical architecture.
I hope that helps to clarify things - thanks to @SteveSmithCK and @harbars for their input on this.
Note: an updated version of this post with revised recommendations is available here.
As the agenda looked so interesting I decided to drive to Ullesthorpe (home of Combined Knowledge in the midlands) on July 22 2010 to the SharePoint User Group UK (#SUGUK). Considering I live in West Berkshire (and drive a 1993 Ford Fiesta, which I am rather fond of) this turned out to be a 5 hour round trip. Was it worth it? Hell yes! In addition to two very different - but fantastic - presentations from @mattgroves (big on social and lead me to believe that SharePoint is in fact an elephant) and @SteveSmithCK (capacity planning extraordinaire), I managed to win myself a 12-month MSDN subscription! Given that the question was basically "who has the rights to deploy a sandboxed solution in SharePoint 2010?", I consider myself very lucky to have won given the number of other SharePoint addicts sat in the room with me. I think the fact that most of the other attendees appeared to already have one (I heard someone behind me talking about how she planned to give her "spares" away) helped considerably.
However, whilst I enjoyed both presentations thoroughly, one question from the audience troubled me somewhat: "is there a limit to the number of application pools you can host on each WFE server in SharePoint 2010"? While Steve's answer was "around 100 before you start to run into IIS issues", I couldn't help but shake a nagging feeling that I had read the TechNet recommended maximum was 10. In fact, I was so damn sure of that TechNet number (10 app pools / WFE) that I would have bet my brand new MSDN subscription on it.
Now I was faced with a dilemma here: did I take Steve Smith's guidance as gospel, or go with Microsoft's recommendation? I did what I figured every sensible SharePoint administrator would do in my situation: I sat down, got back in my box and reminded myself that on the odd occasion TechNet tells more fibs than my parents used to on Christmas eve.
I did however make a point of looking up the TechNet article in question this evening. It does indeed list 10 application pools per Web server as a recommended guideline - in this case the "Limit type" column indicates the figure is a supportability number. Now, I know MS supportability numbers and they are not typically less than you would expect - and I would certainly not expect it to be ten times less than the guidance provided by a SharePoint legend such as Steve Smith. As a recent example, @harbars kindly informed me that having over 50 Web applications in one MOSS farm was, in his words (tweet) "madness" - Microsoft recommend no more than 99 Web applications. I think you know which guidance I paid attention to and he doesn't live in Redmond.
To be fair to Microsoft, the article above does also state that the maximum number is largely determined by hardware capabilities and the workload that the farm is serving. Steve also confirmed this at the user group by telling me that the app pool ceiling has been lifted dramatically due to the move to a 64-bit-only architecture for SharePoint 2010. Now I know that 64-bit brings about numerous performance and scalability benefits (not least of which is a practically unlimited, continuous address space for user mode processes), but I am still keen to find out why there is such a huge discrepancy between Microsoft's stated supportability figure and the guidance of industry experts who have used the product for the best part of 2 years.
So, rather than pester Steve again, I did what I often do when I have a nagging capacity-related query: I sent the query to a few friendly folk in the SharePoint community who (despite the fact that they are quite clearly very, very busy people) normally find the time to respond to my numerous concerns. Did anyone reply this time? You bet.
@ToddKlindt gave me a pretty clear response by saying that "anything Steve says, I believe". Sounds like good advice right? He also told me that the documented MS advice was actually pretty good considering that they suggest the limitation is largely hardware dependant. For those of you that don't know Todd, he is a well respected SharePoint guru and a co-author for Professional SharePoint 2010 Administration. If you don't have it already and you are a SharePoint 2010 administrator I can whole-heartedly recommend that you add this to your one to your collection of geeky paperbacks.
@joeloleson's article entitled "What’s New in SharePoint 2010 Capacity Planning" also mirrors the advice kindly provided by Steve and Todd. Joel points out that Microsoft have adjusted some of their capacity planning guidance based simply on the fact that IIS and SQL scale better with 64-bit, and states that the number of application pools " totally depends on the RAM on the box".
So how many Web applications can you host on a SharePoint 2010 WFE server? It looks like the usual SharePoint consultancy response: "it depends on your hardware's capability and server farm load", and there is no magic number other than the guidance provided by Steve that you may run into IIS issues as you approach 100 pools on one server. Microsoft's supportability number of 10 is there as a rough guide (probably for those running a minimal WFE hardware configuration - think 8GB of RAM), but is almost completely dependent on your available hardware. The new SharePoint 2010 Administration Toolkit includes a load testing kit that should help administrators validate their hardware requirements as part of a capacity planning exercise.
I hope that clears things up a little bit for anyone else concerned that they have more than 10 application pools on their SharePoint 2010 WFE servers. I'll see you at the next #SUGUK!
Having just received our SharePoint 2010 Evolution Conference DVDs, I couldn't resist browsing through the vast quantity of slide decks provided by Combined Knowledge. Spencer Harbar's deck on Multi-tenancy immediately grabbed my attention given that hosting is a core part of what we do.
Multi-tenancy in the context of SharePoint is Microsoft's term for "hosting". In SharePoint 2007 (MOSS), hosting a SharePoint environment with multiple tenants was a significant challenge. Issues included a limitation on the number of Web applications that could be created, site collection management / isolation problems and URL namespace restrictions.
In SharePoint 2010, Microsoft have to a large extent dealt with the last two issues mentioned above. Creating a large number of Web applications within a single farm still causes resource problems so the preferred scaling approach for hosting companies is likely to be to go with site collections - especially considering they can now be partitioned and managed by tenants themselves.
Host-named site collections in MOSS were generally considered a no-go. On paper, they did not support managed paths or SSL termination. In practise, they were fraught with issues and administrators generally avoided using them.
So what's changed for host-named site collections in SharePoint 2010? In short, they are fixed. By this, I mean:
This makes it entirely possible to have "vanity URLs" on a per site collection basis. For example (domain name ownership issues aside), you could have www.microsoft.com for one site collection and www.ibm.com on another - within the same Web application.
I should mention at this point that there is one limitation around host-named site collections that you should be aware of: they do not support alternate access mappings and are therefore always considered to be in the default zone. Additionally, host headers cannot be applied at the Web application (IIS site) level, and doing so makes it impossible to access host-named site collections (IIS ignores requests that do not match the host binding). Read more on Technet.
Prerequisites:
DNS Entries for vanity URLs
OK so I dont imagine setting up a couple of DNS entries is going to be tricky for the IT pros out there. However, I will include this step for those who may not be familiar with DNS administration.
Add new forward lookup zones for your vanity URL domains, e.g. "microsoft.com" and "ibm.com".
Within each lookup zone created above, add a new host, e.g. "www" with an IP for your test server. On my test VPC I just added 127.0.0.1.
DNS entries for vanity URLs
Create Web application
If you don't have an appropriate test Web application, go ahead and create one. Ensure that you do not apply a host header at this stage.
Ensure you don't apply a site level host header
Create host-named site collections using PowerShell
Open up the SharePoint 2010 Management Shell and enter the following commands to create a new site collection for each of your desired vanity URLs. In this case I have created a site collection with the host name www.microsoft.com:
New-SPSite http://www.microsoft.com -OwnerAlias DOMAIN\username -HostHeaderWebApplication http://servername
Entering the PowerShell commands; don't forget the dash before HostHeaderWebApplication!
You should now be able to access your new site collections using the vanity URLs you have specified. You have successfully created multiple host-named site collections within one Web application demonstrating the use of vanity URLs to cater for multi-tenant scenarios in a scalable manner.
All done - your new host-named site collections should be accessible
Host-named site collections are vastly improved in SharePoint 2010 when compared to previous versions. This, combined with the other benefits mentioned above such as improved tenant administration and data partitioning for site collections provides a compelling case for hosting companies to upgrade. If you are interested in multi-tenancy in SharePoint 2010 I would strongly recommend you check out Spencer's post linked at the beginning of this post.
Subscribe to the RSS feed
Follow me on Twitter
Follow my Networked Blog on Facebook
Edit (13/08/2010)
@joeloleoson Expands this article by explaining how vendors use Microsoft’s protocol documentation to "do things not readily exposed in the object model" on his blog. I recommend you have a read.
=========
I have seen a few queries recently around which database operations are supported in SharePoint. As SharePoint administrators, we need to be aware of which operations should be run on farm databases in order to ensure good farm health and performance - and which operations we should avoid at all costs in order to keep databases in a supported state.
Whilst I might take a slightly more relaxed approach in development environments with regard to database operations (I don't expect support from Microsoft for my VPC), my rule in a production environment is clear: never, ever perform an unsupported operation unless specifically asked to by Microsoft support. That includes reading from a content database - which might surprise some.
That's right - reading from a SharePoint DB will leave it in an unsupported state. That's confirmed in this KB article that was updated in April of this year and applies to pretty much any SharePoint product since WSS 2.0 up to and including SharePoint Server 2010 (with the exception of the logging database, where customisations are fully supported).
If you a take moment and think about this, prohibiting changes to the DB schema actually makes a lot of sense. Microsoft tests SharePoint based on a specific database schema and could not possibly predict changes that third parties are likely to make. Therefore, it could not practically support those changes. I think of this as analogous to purchasing a new BMW Z4, fitting a supercharger and then expecting the dealer to honour any warranty they may have promised - it just would not be practical nor economically viable.
As regards to preventing read operations (e.g. SELECT statements), I suppose that it's analogous to breaking the warranty sticker on an electrical appliance. OK, you may not have touched anything but it implies you may have done. Similarly, if Microsoft discover that you have been taking a look through a content database they would be inclined to think you have made a few changes along the way.
So we know that reading from a SharePoint database is prohibited. Here are a few other operations that Microsoft specifically list in the above KB article as leaving a database in an unsupported state:
• Adding database triggers
• Adding new indexes or changing existing indexes within tables
• Adding, changing, or deleting any primary or foreign key relationships
• Changing or deleting existing stored procedures
• Calling existing stored procedures directly
• Adding new stored procedures
• Adding, changing, or deleting any data in any table of any of the databases for the products that are listed in the "Applies to" section
• Adding, changing, or deleting any columns in any table of any of the databases for the products that are listed in the "Applies to" section
• Making any modification to the database schema
• Adding tables to any of the databases for the products that are listed in the "Applies to" section
• Changing the database collation
• Running DBCC_CHECKDB WITH REPAIR_ALLOW_DATA_LOSS (However, running DBCC_CHECKDB WITH REPAIR_FAST and REPAIR_REBUILD is supported, as these commands only update the indexes of the associated database.)
I recommend you read the KB article linked above for the most up to date list of (un)supported operations.
So which operations can we perform on a SharePoint database? The SQL DBA's out there will know that routine maintenance is essential for optimum performance - and few areas are as important to SharePoint performance than the SQL back end. Whilst I have not yet seen an equivalent resource for SharePoint 2010, Microsoft published a very useful white paper back in November 2009 called "Database Maintenance for Office SharePoint Server 2007". I encourage you to take a look if you haven't seen this already and are involved in MOSS administration (although I imagine it will be relevant to SharePoint 2010 also).
According to this white paper, the following maintenance tasks are recommended:
• Checking database integrity (ensures that your data and indexes are not corrupted).
• Defragmenting indexes by either reorganizing them or rebuilding them (applies mainly to indexes; fragmentation can be reduced by setting a large database start size).
• Setting the fill factor for a server (allows index performance to be fine tuned).
• Shrinking databases to recover unused disk space (white space caused by deleting documents, libraries, lists etc.).
Note that Microsoft do not recommend configuring auto-shrink operations, and advise only to shrink content databases due to the fact that other types of databases (e.g. SSP) do not typically undergo a large number of deletions.
In conclusion, things haven't really changed much since MOSS with regard to which DB operations are supported - the main message is to keep well away from them! I imagine Microsoft will release a similar resource for SharePoint 2010 with an updated list of recommended DB maintenance operations. For now, I will be implementing the above MOSS maintenance plans on my SharePoint 2010 test environment.
