Why we are using warm-up scripts

SharePoint is built upon the Microsoft .NET Framework and ASP.NET, specially the 2.0 releases of those technologies. ASP.NET is architected so that its pages are not compiled until the first time a server request .As you know when an IIS worker process has coming up for the first time , It makes lots of jobs in the background. The big portion of this time used by compiling assemblies, and reading configurations  . For a standard ASP.Net application it takes very small time but for a SharePoint Application we have tons of assemblies and configurations and more than one WCF services which is need to be activated . So it will usually takes more time over 1 minutes. The warm up duration also depends on your configuration , hardware and assembly counts and changes for every system.

The issue happens when application pool recycle, iisreset or a worker process crashes.  And for SharePoint sites every application pool has a configuration that force the recycle itself . You can check from IIS -> Application Pools -> (For example Select “SharePoint Central Administration v4”) and right click   Select Recycling option And you can see the recycle time.

We have also an Idle Time Out property defined in Application Pool settings that make free worker processes resources which are in idle state for defined time period . Configure Idle Time-out Settings for an Application Pool (IIS 7) http://technet.microsoft.com/en-us/library/cc771956(v=WS.10).aspx

However even it is disabled by setting “Idle Time Out” value to 0 still IIS requires and also suggested way that recycle application pools daily manner. The default IIS behavior of naturally cycling an app pool somewhere between 1am and 3am or off hours if there is such a thing is healthy to clean out the memory space in IIS. Also same consideration is valid for SharePoint Timer service , there is a Timer job in SharePoint for responsible to restart timer service in daily manner. It is a good and suggested thing http://blogs.msdn.com/b/besidethepoint/archive/2012/01/10/the-timer-recycle-job-job-timer-recycle.aspx

But sometime in the midnight a recycle occurs and assume that no one send a request a server until morning and when a user open his browser and try to connect SharePoint site He/She should wait more, because the worker process is only getting up until the first time a server get the request. Warm up scripts send a requests to your server for a scheduled time  that forces IIS to worker process up and running .when a user’s request a page through a browser, preventing your users from seeing a delay when making that first request. That’s pretty much all they do, and the only purpose most of them serve is to “warm up” your server by having precompile your site’s ASP.NET pages ahead of an actual user request. You can use warm up scripts .

For the warm up scripts which is described in fallowing article http://blogs.technet.com/b/rgullick/archive/2011/12/02/minimalist-sharepoint-warmup-script.aspx have very common usage on this kind of issues.

 

SharePoint 2010 Form based authentication problem Event ID:1315 and Event ID:8306

Assume that you have a SharePoint 2010 site with configured as Claim Based Authentiction with custom SQL Membership . You have move your site and membership database to another server and you have facing with connection problems on existing SQL MemberShip users by getting this fallowing errors

In ULS Logs:

11.13.2012 15:40:18.20 w3wp.exe (0x18C8) 0x17C8 SharePoint Foundation Claims Authentication 0000 Unexpected Password check on ‘user@mail.com’ generated exception: ‘System.ServiceModel.FaultException`1[Microsoft.IdentityModel.Tokens.FailedAuthenticationException]: The security token username and password could not be validated. (Fault Detail is equal to Microsoft.IdentityModel.Tokens.FailedAuthenticationException: The security token username and password could not be validated.).’.

11.13.2012 15:40:18.20 w3wp.exe (0x18C8) 0x17C8 SharePoint Foundation Claims Authentication fo1t Monitorable SPSecurityTokenService.Issue() failed: System.ServiceModel.FaultException`1[Microsoft.IdentityModel.Tokens.FailedAuthenticationException]: The security token username and password could not be validated. (Fault Detail is equal to Microsoft.IdentityModel.Tokens.FailedAuthenticationException: The security token username and password could not be validated.).

and In Event logs
Presence of Event ID 8306 in the Application Event Log
11/08/2012 03:29:11 PM Error SERVERA 8306 Microsoft-SharePoint Products-SharePoint Foundation Claims Authentication DOMAIN\User An exception occurred when trying to issue security token: The security token username and password could not be validated..

Presence of Event ID 1315 in the Application Event Log with Event code: 4006
Event message: Membership credential verification failed.

The problem is here if you try to login site with one of existing FBA user even password is correct , cound not able to validate password . If you create a new FBA user , there is no problem on login.
The main cause of this issue could be changes of the Machine Key.

Why ?

The Password information is stored in the aspnet_Membership table in Asp.Net Membership database . The   SqlMembershipProvider allows for passwords to be stored in the database    using one of the following three techniques:

  • Clear – the password is stored in the database as plain-text. I strongly        discourage using this option. If the database is compromised – be it by a hacker        who finds a back door or a disgruntled employee who has database access – every        single user’s credentials are there for the taking.
  • Hashed – passwords are hashed using a one-way hash algorithm and a randomly        generated salt value. This hashed value (along with the salt) is stored in the database.
  • Encrypted – an encrypted version of the password is stored in the database

The password storage technique used depends on the SqlMembershipProvider    settings specified in Web.configThe default behavior is to    store the hash of the password.
the particular encryption or hashing algorithm used by the SqlMembershipProvider is determined by the settings in the <machineKey> element.

for more information:
http://www.asp.net/web-forms/tutorials/security/membership/creating-the-membership-schema-in-sql-server-vb
http://www.asp.net/web-forms/tutorials/security/introduction/forms-authentication-configuration-and-advanced-topics-vb

So if you have move your site to another server you may consider that the MachineKey if anyhow is changed , the existing users’ passwords can not be validated.

1) First Check for the MachineKey values in web.config for related your FBA SharePoint site. If you have any difference on target site make them equalize.
2) Also don’t forget to check other servers in your farm for the same site should be same MachineKey. If any difference in MachineKeys may cause integrity problems.

Somehow If the data integrity has broken , recreating users or forcing the users reset their password will help about the issue.

 

SharePoint Slow Warm-Up Problem

It is a well known issue that  if you have a SharePoint and after you recycle the application pool of site or make an iisreset or any movement that force to recycle (web.config modifications etc.),When the first request send to the server , the response of the server is very slow for the first time like 2 minutes. After this first slow reponse ,your server begins responding very quickly. We are calling this duration as warm up state.
So what’s happening in warm up ? In warm up state basically your worker process are getting up, while a worker process is getting up, the necessary assemblies are compiled and loaded , and needed configuration files are read and prepare the process for serving  the requests. If your web site has integrated with other sites in same server also may need to warm up sequence for other worker process and the process of our site can wait other process’s warmup to create a reliable response to you. And expected that all this operations could take some time that is depended on your assembly count, configuration, hardware etc.

But what if it is taking more than usual ? For example assume  that you have identical sharepoint site hosting in identical two farm and the warm up times are different like first instace of the site hosted server A have 20-30sec warm up and second instance of the site hosted in Server B it takes 180seconds.. So there should be a problem here.

This problem occurs because a .NET Framework 2.0 managed assembly that has an authenticode signature takes longer than usual to load.when loading signed assemblies the .net Framework 2.0 checks the Internet based Certificate Revocation list for verifing these signatures and the signatures is always verified when the .NET Framework 2.0 managed assembly that has an authenticode signature is loaded .And SharePoint has lots of signed assemblies.For some environment if does not have any Access to internet it causes a delay like 2 minutes to respond .

If you collect a process monitor log from your worker process , you may face the time gap bettween fallowing context.

14:57:39,6298324 w3wp.exe 3648 TCP Reconnect SERVERB.postman.local:63017 -> 80-239-149-10.customer.teliacarrier.com:http SUCCESS Length: 0, seqnum: 0, connid: 0 14:57:46,6170930 w3wp.exe 3648 RegCloseKey HKU\S-1-5-21-220523389-842925246-682003330-29313\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections SUCCESS

0 ntoskrnl.exe CmpCallCallBacks + 0x1c0 0xfffff80001a25fc0 C:\Windows\system32\ntoskrnl.exe
1 ntoskrnl.exe  ?? ::NNGAKEGL::`string’ + 0x364df 0xfffff800019c28f6 C:\Windows\system32\ntoskrnl.exe
2 ntoskrnl.exe ObfDereferenceObject + 0xd4 0xfffff8000168a3d4 C:\Windows\system32\ntoskrnl.exe
3 ntoskrnl.exe ObpCloseHandleTableEntry + 0xc4 0xfffff80001973b94 C:\Windows\system32\ntoskrnl.exe
4 ntoskrnl.exe ObpCloseHandle + 0x94 0xfffff80001974144 C:\Windows\system32\ntoskrnl.exe
5 ntoskrnl.exe KiSystemServiceCopyEnd + 0x13 0xfffff80001680453 C:\Windows\system32\ntoskrnl.exe
6 ntdll.dll ZwClose + 0xa 0x7748140a C:\Windows\System32\ntdll.dll
7 kernel32.dll RegCloseKey + 0xbc 0x77324cd0 C:\Windows\System32\kernel32.dll
8 winhttp.dll CRegBlob::~CRegBlob + 0x1c 0x7fef98fb768 C:\Windows\System32\winhttp.dll
9 winhttp.dll ReadWinInetProxySettings + 0x1fb 0x7fef990a9f7 C:\Windows\System32\winhttp.dll
10 winhttp.dll WinHttpGetIEProxyConfigForCurrentUser + 0x28a 0x7fef990a674 C:\Windows\System32\winhttp.dll
11 cryptnet.dll InetGetProxy + 0x11e 0x7fef69eae42 C:\Windows\System32\cryptnet.dll
12 cryptnet.dll InetSendAuthenticatedRequestAndReceiveResponse + 0x190 0x7fef69e9237 C:\Windows\System32\cryptnet.dll
13 cryptnet.dll InetSendReceiveUrlRequest + 0x57e 0x7fef69e983d C:\Windows\System32\cryptnet.dll
14 cryptnet.dll CInetSynchronousRetriever::RetrieveObjectByUrl + 0xc4 0x7fef69e9d9c C:\Windows\System32\cryptnet.dll
15 cryptnet.dll InetRetrieveEncodedObject + 0xbf 0x7fef69e29c5 C:\Windows\System32\cryptnet.dll
16 cryptnet.dll CObjectRetrievalManager::RetrieveObjectByUrl + 0x15c 0x7fef69e2ef0 C:\Windows\System32\cryptnet.dll
17 cryptnet.dll CryptRetrieveObjectByUrlWithTimeoutThreadProc + 0xc1 0x7fef69e9acc C:\Windows\System32\cryptnet.dll
18 kernel32.dll BaseThreadInitThunk + 0xd 0x7732652d C:\Windows\System32\kernel32.dll
19 ntdll.dll RtlUserThreadStart + 0x1d 0x7745c521 C:\Windows\System32\ntdll.dll

 

This proves that the CRL check causes delay. For a resolution you can disable CRL check on .net level

You can find more detail and how to fix it this problem in following KB Article. http://support.microsoft.com/kb/936707

This hotfix adds the generatePublisherEvidence configuration setting to the .NET Framework 2.0. After you apply this hotfix, you can use this configuration setting to disable signature verification in a .NET Framework 2.0 managed application.  You can use this configuration setting in an application configuration file.  To do this, add the following code to the <ApplicationName>.exe.config file for the .NET Framework 2.0 managed application:

<configuration>
	<runtime>
		<generatePublisherEvidence enabled="false"/>
	</runtime>
</configuration>

If your application is hosted in IIS, change one of the following:

  • C:\Windows\Microsoft.NET\Framework\v2.0.50727\aspnet.config
  • C:\Windows\Microsoft.NET\Framework\v2.0.50727\CONFIG\machine.config

NoteOn x64 machines, you must also change one of the following:

  • C:\Windows\Microsoft.NET\Framework64\v2.0.50727\aspnet.config
  • C:\Windows\Microsoft.NET\Framework64\v2.0.50727\CONFIG\machine.config

Sharepoint 2007/2010 October CU 2012 has released.

WSS V3 and MOSS 2007 October 2012 CU:
http://blogs.msdn.com/b/joerg_sinemus/archive/2012/11/12/wss-and-moss-october-2012-cu.aspx
http://blogs.technet.com/b/stefan_gossner/archive/2012/11/08/october-2012-cu-for-sharepoint-2007-has-been-released.aspx

MSF2010 and SPS2010 October 2012 CU:
http://blogs.msdn.com/b/joerg_sinemus/archive/2012/11/15/sharepoint-2010-and-october-2012-cu.aspx
http://blogs.technet.com/b/stefan_gossner/archive/2012/11/15/october-2012-cu-for-sharepoint-2010-has-been-released.aspx