Sharepoint Optimization Thread

[TheOracle]

Ok, since there are so many threads that have mentioned sharepoint, I wanted to start one with some optimization policies, and to get discussions going on them.  I'll start with some safe policies, and add to them tomorrow.

As the response headers from sharepoint contain many useless HTTP headers, which can increase the amount of non-compressible content in each response, here is a set of policies to remove them.  From initial testing this doesn't seem to cause any problems, but if anybody has a reason not to remove these headers, just let me know:

Bind the following globally or on a vserver as appropriate:

add rewrite action remove_server delete_http_header Server
add rewrite policy remove_server true remove_server

add rewrite action remove_x-powered delete_http_header X-Powered-By
add rewrite policy remove_x-powered true remove_x-powered

add rewrite action remove_mspts delete_http_header MicrosoftSharePointTeamServices
add rewrite policy remove_mspts true remove_mspts

add rewrite action remove_x-asp delete_http_header X-AspNet-Version
add rewrite policy remove_x-asp true remove_x-asp

add rewrite action remove_accept-ranges delete_http_header Accept-Ranges
add rewrite policy remove_accept-ranges true remove_accept-ranges

add rewrite action remove_content-location delete_http_header content-location
add rewrite policy remove_content-location true remove_content-location

add rewrite action remove_etag delete_http_header Etag
add rewrite policy remove_etag "http.RES.HEADER(\"Last-Modified\").EXISTS&&http.RES.HEADER(\"Etag\").EXISTS" remove_etag

# adjust the rewrite path to point to the default people would get redirected to anyway, avoids a redirect on initial connect
add rewrite action root_handler replace http.REQ.URL.PATH "\"/Default.aspx\""
add rewrite policy portal_root "http.REQ.URL.PATH.EQ(\"/\")" root_handler

[TheOracle]

Next trick:  tuning IC.  As Sharepoint attempts to include good cache-control headers for each object for optimal caching, the following is a good set of policies that can be bound.  As IC doesn't let you bind policies (yet) to vservers, you need to either pin these by destination IP and/or host header.  I'll use host header for this example (on the domain portal.domain.com):

set cache contentGroup DEFAULT -heurExpiryParam 10 -insertVia NO -insertETag NO -maxResSize 2048
add cache policy not_portal_not_cache -rule "REQ.HTTP.HEADER Host != portal.domain.com" -action NOCACHE
bind cache global not_portal_not_cache -priority 100 -precedeDefRules YES

As any request with the proper domain will hit the default rules, it will follow RFC behavior, and cache fairly well.  Combined with the rewrite policies in the prior post, you will get better results as well due to the cleanup of some of the headers.

[TheOracle]

Sharepoint Persistence:

Use Cookie Insert, with a session timeout of 0.  This allows the persistence to follow the session, and prevents set-cookie headers from being issued in every response to update the timeout on the cookie.

Service behavior:  Enable TCPB, CMP

Global Behavior:  enable SACK, window scaling

[TheOracle]

Compression:

For compression, I'm going to recommend an aggressive set of policies that may be a little controversial, in that it may not work completely correctly at first:

add cmp policy ns_nocmp_multimedia -rule "RES.HTTP.HEADER Content-Type == \'image/*\' || RES.HTTP.HEADER Content-Type == \'audio/*\' || RES.HTTP.HEADER Content-Type == \'video/*\' || RES.HTTP.HEADER Content-Type == \'multipart/*\'" -resAction NOCOMPRESS
add cmp policy ns_nocmp_compressed -rule "RES.HTTP.HEADER Content-Type CONTAINS zip || RES.HTTP.HEADER Content-Type == \'application/java-archive\'" -resAction NOCOMPRESS
add cmp policy compress_get_post -rule "REQ.HTTP.METHOD == GET  || REQ.HTTP.METHOD == POST " -resAction COMPRESS
bind cmp global ns_nocmp_multimedia -priority 100
bind cmp global ns_nocmp_compressed -priority 200
bind cmp global compress_get_post -priority 300

In effect, if the content is
  a multimedia type (image/*, audio/* video/* or multipart/*) don't compress;
  Known compressed types, don't compress
  Otherwise, compress, override all other controls in the NS.

This set of policies though should be understood before being applied:

1.  The server is assumed to be applying proper mime types to the multimedia and compressed content.  In many cases, the mime types arn't associated with the extensions properly, and the result is the default mime type of text/plain, resulting in compression occurring on objects that shouldn't be compressed.  The "right" way to fix this is to fix the server, as this can cause other subtle problems as well when implementing compression, as IE processes extensions and mime types differently when compression is involved.  Fixing the mime type is in general "better" than trying to use the extensions directly to control compression, although using extensions is easier.
2.  IE has bugs with compression.  http://www.netscalerkb.com/netscaler_qa/problemas_with_owa-t200.0.html;msg238#msg238.  If turning on causes an issue with any behavior, then the issue is most likely a bug in the browser.  Exceptions are built-into the NS to not compress XML for IE and for Netscape 4.7, however, in reality, chances are nobody trying to use sharepoint will be using Netscape 4.7.  IE issues in general need to be handled as exceptions, as it is usually a combination of several factors that cause the behavior, not just one factor such as XML content types.
3.  The policy does not exclude pdf and flash files, both of which **could** be internally compressed.  Often enough though, they are not properly encoded to compress the data internally.  In addition, even if compressed internally, there is generally enough compressible framing information so that the extra layer of compression won't expand the content.  As such, I'm not suggesting excluding these file formats from compression.

[nzambo]

Thanks for the tips!  The headers exclusions are great.  It turns out that some of the headers contained information about internal network resources that we'd rather not be shipping out.

I have a question about the integrated caching... how can we tell it's working?  The content group shows no mem usage and no hits, but the policy shows plenty of hits.  Is this something that really doesn't kick in until there's a decent amount of throughput?

Nick

[TheOracle]

Thanks for the tips!  The headers exclusions are great.  It turns out that some of the headers contained information about internal network resources that we'd rather not be shipping out.

I have a question about the integrated caching... how can we tell it's working?  The content group shows no mem usage and no hits, but the policy shows plenty of hits.  Is this something that really doesn't kick in until there's a decent amount of throughput?

Nick

You should see hits on the default policy and mem usage.  Is the global cache params set with the proper amount of memory (it can get set to 0 in various situations, preventing caching)?  If not, is the content being authenticated, and if so, what method of authentication are you using.  I'm trying to get access to a sharepoint install that isn't via VPN so I can test various aspects of this and how it interacts with authentication headers, so that is one area that could be messing up deals with the authentication, as IC+authentication is generally a bad idea, as IC won't be able to validate that a user should be able to get an object.  In theory, the IC default policies won't cache objects that require authentication, but I'm seeing if there is a way to work around this that is fairly clean.

[nzambo]

duh.. global memory setting = 0

You rock...

Now we're setting memory usage.. not much, but then again we're just hitting the site a few times. Still no non-304/304 hits, but plenty of hits on the cache policies.

[TheOracle]

You probably won't see 304's for a while--if you check the headers that sharepoint inserts, it inserts very long cache-control: max-age values for any content, to no if-modified has to be inserted.  The NS will also honor it, so will cache the objects for a good long time if not expelled from the cache for memory.

The Oracle

[nzambo]

Thanks as always

[TheOracle]

Does anybody have a sharepoint site using NTLM on a public IP that I can work with them?  It is apparent that with NTLM some tuning needs to be done, and in order to develop a set of best practices for this situation, I'll need to do some testing, but I don't have a server running sharepoint to test on.  If you have a setup (NS running 8.1 and sharepoint with NTLM) that I can work with for a few hours, can you drop me a note via the site mail, and we can exchange info to set this up.  Thanks!

The Oracle

[evildani]

If you want I can start installing it right now and finish by mid afternoon. I will get on to it.

[evildani]

Site ready and NS ready, mail me and I will send you the info to login.

[TheOracle]

A light turns on:  http://technet2.microsoft.com/windowsserver/en/library/35d4445b-5440-4dc8-80f3-cea51b23abbc1033.mspx?mfr=true

When using NTLM authentication with IIS6, every request will by default result in an NTLM authentication occuring, which adds two request/response pairs before even a 302 can be received for an object.  This obviously will kill performance for content that doesn't allow anonymous access, which is why I was observing such bad times on my initial testing with sharepoint.  I'll be checking into how to change this behavior, but in such a way to prevent security issues from occurring.

Erik

[TheOracle]

I'll be reposting the configuration items to use--there are some changes to the previously posted that are fairly major, in particular with compression.  Turns out a * when using == to match header values won't work, so we need to use contains instead without a wildcard.

The Oracle

[TheOracle]

one new optimization point that may be of use to anybody working with sharepoint:

nsapimgr -B "call ns_cache_set_req_check_authorization(0)"

This will allow the netscaler default IC configuration to ignore the fact that authentication occured on a connection/request, and allow the response to be cached based on other headers, i.e. a cache-control max-age header.  This is very tricky to do in a secure manner, as once an object is cached, it can be requested and served without authentication from the IC engine.

The Oracle