I am at my wit's end trying to figure out the following problem:
We have an SP2013 SP1 installation with a fairly standard setup. About a month or so ago, both of the WFEs began throwing the exact same errors every 2 to 3 minutes. The Event IDs are 7024 and 7031. 7024 indicated that the timer service had terminated with a server-specific error of Unspecified error. 7031 basically reiterated that by telling me the service had terminated unexpectedly, how many times it had done so and when the next corrective action would take place. After a few minutes the Timer service restarts itself, runs for 2 or 3 minutes and then deja vu.
In the ULS logs I have found the following sequence of events that occur at the same time that the 7024 and 7031 errors are being thrown:
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 72au Medium Service Microsoft.Office.Server.Administration.ProfileSynchronizationService, id 7eb34ace-9630-4c52-96aa-263d723b6f88 not online, ignoring
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 72au Medium Service Microsoft.SharePoint.Administration.SPRequestManagementService, id 4256b1e3-faa5-49ed-852b-22d28d1bab34 not online, ignoring
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 72au Medium Service Microsoft.Office.Server.Search.Administration.SearchAdminWebService, id bf93e7a2-a6d0-4844-b28f-ef3ce9e80c70 not online, ignoring
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 72au Medium Service Microsoft.SharePoint.Administration.Claims.SPWindowsTokenService, id 8088d56c-f5c8-4cc3-826d-7324b3651579 not online, ignoring
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 72au Medium Service Microsoft.PerformancePoint.Scorecards.BIMonitoringService, id 74e0e38c-26ea-4a4d-84c2-2d5bea9db53d not online, ignoring
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 2n2p Monitorable The following error occured while trying to initialize the timer: System.NullReferenceException: Object reference not set to an instance of an object. at System.Object.GetType()
at Microsoft.SharePoint.Administration.SPTimerStore.GetLocalProvisionedServices(Boolean cacheOnlineInstances) at Microsoft.SharePoint.Administration.SPTimerStore.InitializeTimer(Int64& cacheVersion, Object& jobDefinitions, Int32&
timerMode, Guid& serverId, Boolean& isServerBusy) at Microsoft.SharePoint.Administration.SPNativeConfigurationProvider.InitializeTimer(Int64& cacheVersion, Object& jobDefinitions, Int32& timerMode, Guid& serverId,
Boolean& isServerBusy)
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 7v43 Medium An error occured while initializing the timer.
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 5utx Unexpected The timer service could not initialize its configuration, please check the configuration database. Will retry later.
12/07/2015 21:32:15.18 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer g1ms Unexpected Exiting the process because the timer could not be initialized after multiple attempts.
12/07/2015 21:32:15.21 OWSTIMER.EXE (0x1BF0) 0x11E0 SharePoint Foundation Timer 8gs7 Medium The timer service is stopping
12/07/2015 21:32:16.86 OWSTIMER.EXE (0x1BF0) 0x07AC SharePoint Foundation Unified
Logging Service 8wsw High Now terminating ULS (OWSTIMER.EXE, onetnative.dll)
I have either verified or done the following:
Verified that service account timer service is running under has proper permissions
Verified that account has proper permissions to config db in sql
Cleared out the config cache, and reset cache.ini file
Rerun spconfig successfully
Couple of things to note:
This appears to have started soon after an attempt to update to Nintex Workflows 2013 was unsuccessful. I say appears because we unfortunately don't have space to store logs for longer than a couple of weeks so I can't go back and confirm that; I'm relying upon my own memory recall for that.
In CA, neither WFE has any timer jobs scheduled to be run
Any help anyone could offer would be most most appreciated.