We have Sharepoint Server 2013 installed on a AD domain controller and Office Web Apps installed on a machine in the same domain and configured for internal-http WOPI zone. Discovery XML works fine. When opening a MS Office document, Office Web Apps show
its label for a minute or two, and then display the message "Sorry, Web App can't open this document because the service is busy. Please try again later" or the message "Web App cannot open this document due to an unknown error."
Network logs show repeated requests to Docdatahandler.ashx to which Web Apps server each time responds with some XML saying it's busy, and to Reshandler.ashx to which the server each time responds with 404 status.
The ULC logs for Sharepoint server look like this:
Mapping URI from 'http://sharepoint:80/_vti_bin/wopi.ashx/files/…/contents?access_token=…&access_token_ttl=1403904071991' to 'http://sharepoint/_vti_bin/wopi.ashx/files/…/contents?access_token=…&access_token_ttl=1403904071991'
Entering monitored scope (Application Authentication Pipeline). Parent Request (GET:http://sharepoint:80/_vti_bin/wopi.ashx/files/…/contents?access_token=…&access_token_ttl=1403904071991)
Entering monitored scope (Getting Site Subscription Id). Parent [Loopback] Getting token from STS and setting Thread Identity
Leaving Monitored Scope (Getting Site Subscription Id). Execution time=0,0601117711447749
Entering monitored scope (Reading token from Cache using cacheKey claim value). Parent [Loopback] Getting token from STS and setting Thread Identity
Leaving Monitored Scope (Reading token from Cache using cacheKey claim value). Execution time=2,55094406980315
SPApplicationAuthenticationModule Authentication finished successfully for user:0#.w|vserver\0347 and actor:
Leaving Monitored Scope (Application Authentication Pipeline). Execution time=4,30284782797009
OAuth request. IsAuthenticated=True, UserIdentityName=0#.w|vserver\0347, ClaimsCount=21
WOPI new request (GetFile), url: http://sharepoint/_vti_bin/wopi.ashx/files/…/contents?access_token=… CALLER-[machine: OfficeWebAppsServer, version: 15.0.4481.1000, CorrelationId:]
Site=/
WOPI Proof: Missing header [X-WOPI-ProofOld] for url: http://sharepoint/_vti_bin/wopi.ashx/files/…/contents?access_token=… CALLER-[machine: OfficeWebAppsServer, version: 15.0.4481.1000, CorrelationId:]
WOPI (GetFile) Proof Data: AccessToken Hash '-2012506731' [1012 bytes], URL 'HTTP://sharepoint/_VTI_BIN/WOPI.ASHX/files/…/CONTENTS?ACCESS_TOKEN=…' [1167 bytes], TimeStamp '635394648765830712' [8 bytes] CALLER-[machine: OfficeWebAppsServer, version: 15.0.4481.1000,
CorrelationId:]
WOPI Signature verification attempt succeeded with public key … [0 ms]
WOPI - InitAndValidateForFile Success [web: http://sharepoint, file: test/ы.docx]
UserAgent not available, file operations may not be optimized. at Microsoft.SharePoint.SPFileStreamManager.CreateCobaltStreamContainer(SPFileStreamStore spfs, ILockBytes ilb, Boolean copyOnFirstWrite, Boolean disposeIlb)
Spent 0 ms to bind 20365 byte file stream
WOPI GetFile - Completed http://sharepoint/_vti_bin/wopi.ashx/files/…/contents?access_token=… CALLER-[byes:20365, machine: OfficeWebAppsServer, version: 15.0.4481.1000, CorrelationId:]
WOPI PerfTrace: <o n="PreWopi" d="6" /><o n="ParseGetFile" d="0" /><o n="InitAndValidateForFile" d="8" /><o n="OpenBinaryStream" d="2" /><o n="WriteResponseStream"
d="17" />
WOPI new request End [Elapsed Time WOPI (ms): 35] [Elapsed Time HttpContext (ms): 42]
Leaving Monitored Scope (Request (GET:http://sharepoint:80/_vti_bin/wopi.ashx/files/…/contents?access_token=… Execution time=42,173001152361
Leaving Monitored Scope (Request (GET:http://sharepoint/_vti_bin/wopi.ashx/files/…/contents?access_token=… Execution time=44,9670172749155
And the ULC logs for Office Web Apps server look like this:
WAC Server HttpModule: Beginning request at target [/wv/wordviewerframe.aspx?ui=ru%2DRU&rs=ru%2DRU&WOPISrc=http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…&sc=http%3A%2F%2Fsharepoint%2Ftest%2FForms%2FAllItems%2Easpx] in session [044adcdc-0567-4727-8d82-2adb93efb266]
ReadStructuredDataFromXml: [C:\Programs\OfficeWebApps\AgentManager\LanguageMapping.xml]
Registering OnPreProcessTrace handler
WOPI Proof Keys - WS Farm State has no Old key (this is expected if recently installed)
WOPI Proof Keys - WS Farm State has no Old key (this is expected if recently installed)
WOPI Proof: Using Current private key (to sign) that matches Current public key …
Updated WOPI Proof Keys [Discovery] - Old Key (Public):
Updated WOPI Proof Keys [Discovery]- Current Key (Public): …
WOPI CheckFileInfo: Start [url:http://sharepoint/_vti_bin/wopi.ashx/files/…]
URL generated for WOPI CheckFile Request: http://sharepoint/_vti_bin/wopi.ashx/files/…?access_token=REDACTED_1040&access_token_ttl=1403904071991
WOPI Proof Data: AccessToken Hash '-2012506731' [1012 bytes], URL 'HTTP://sharepoint/_VTI_BIN/WOPI.ASHX/files/…?ACCESS_TOKEN=REDACTED_1040&ACCESS_TOKEN_TTL=1403904071991' [1158 bytes], TimeStamp '635394648734348738' [8 bytes]
WOPI Proof: Using Current private key (to sign) that matches Current public key …
WOPI Proof - Successfully signed the data with the Current private key [17 ms]
WOPI Proof Keys - WS Farm State has no Old key (this is expected if recently installed)
Cannot create WOPI Proof Token - could not find valid Old proof key data
HttpRequestAsync (WOPICheckFile,WACSERVER) Starting [mode: GET, url: http://sharepoint/_vti_bin/wopi.ashx/files/…?access_token=REDACTED_1040&access_token_ttl=1403904071991]
HttpRequestAsync (WOPICheckFile,WACSERVER) End Called
HttpRequestAsync::GetResponseCallback - ContentLength:2130
HttpRequestAsync::FProcessReadReturn - cbRead:2130
HttpRequestAsync::FProcessReadReturn - cbRead:0
HttpRequestAsync (WOPICheckFile,WACSERVER) Setting Completion [Time in ms: 116, Bytes Read: 2130]
HttpRequestAsync (WOPICheckFile,WACSERVER) SetCompletion Track start | setting trackers | WebRequest.Create() | Create() returned | _req.ContentLength 0 | setting request headers | setting user agent | setting keep alive | setting timeout callback | Start calling
StartResponseProcessing | StartResponseProcessing._req.BeginGetResponse() | BeginGetResponse() returned | StartResponseProcessing RETURNS | Start RETURNS | End.AsyncWaitHandle.WaitOne() | GetResponseCallback isSync:False | _req.EndGetResponse() | EndGetResponse()
returned | RecordResponse OK | _response.GetResponseStream() | _response.GetResponseStream() RETURNED | DoReadLoopFinished | _getStream.BeginRead() | BeginRead() -> sync:False | DoReadLoopFinished RETURNS False | GetResponseCallback DoReadLoopFinished ->
FALSE | GetResponseCallback RETURNS | ReadCallback.FProcessReadReturn() | FProcessReadReturn._getStream.EndRead() | EndRead() -> 2130 | FProcessReadReturn._responseStream.Write() | FProcessReadReturn RETURNS True | FProcessReadReturn() -> TRUE | ReadCallback.DoReadLoopFinished()
| DoReadLoopFinished | _getStream.BeginRead() | ReadCallback.sync returning | BeginRead() -> sync:True | DoReadLoopFinished.FProcessReadReturn() | FProcessReadReturn._getStream.EndRead() | EndRead() -> 0 | FProcessReadReturn._responseStream.Write() |
FProcessReadReturn RETURNS False | FProcessReadReturn() -> false | DoReadLoopFinished RETURNS true | DoReadLoopFinished() -> TRUE | SetCompletion False | WaitOne() returned |
WOPI Http Request Completed [host machine name:sharepoint, version:15.0.4420.1017, host correlation:044adcdc-0567-4727-8d82-2adb93efb266]
Returned from WOPI CheckFileInfo, [host:WACSERVER, user:2b80a4cac8ac4ba9aaeda4b2d11c1c2e]
CacheKey based on host-provided hash: 2f8b6f928ff724d93c1151fb2430b9d837f1068c30654cc5b72131df0796a8de
RoamingAppsAllowList config is empty. Roaming is disabled.
Librarian.BeginGetItem(http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…, Png, docdata.xml)
ViewStore.ReadMainCache(http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…, Png, docdata.xml)
FEMemoryCache.FReadFromCacheLine: key 2f8b6f928ff724d93c1151fb2430b9d837f1068c30654cc5b72131df0796a8de_00000000-0000-0000-0000-000000000602, length 10240, offset 0
GlobalListLoadBalancer: Refreshing list for service DiskCache. Reason: null hasher
GlobalListLoadBalancer: Creating consistent hasher for service DiskCache with 50 nodes.
GlobalListLoadBalancer: Next refresh time: 06/27/2014 11:22:14
GlobalListLoadBalancer: Finished refreshing list. [Agent: DiskCache; Time WriteLock held (ms): 15]
Creating channel to DiskCache: http://OfficeWebAppsServer:809/diskcache/DiskCache.svc
MachineTransfer [SourceMachine:OfficeWebAppsServer, SourceMachineVersion:15.0.4481.1000, TargetMachine:OfficeWebAppsServer, TargetMachineVersion:15.0.4481.1508, Method:DiskCacheService.ReadCacheData]
ReadCacheData: [2F8B6F928FF724D93C1151FB2430B9D837F1068C30654CC5B72131DF0796A8DE00000000000000000000000000000602, 00000000-0000-0000-0000-000000000602, 1048576, 0, OfficeWebAppsServer]
WACServer HttpModule: Completed request with status code [200]
WAC Server HttpModule: Beginning request at target [/wv/ResReader.ashx?n=p1.img&WOPIsrc=http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…] in session […]
WAC Server HttpModule: Beginning request at target [/wv/docdatahandler.ashx?WOPIsrc=http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…]
DiskCacheReader: TimeoutException [Machine: http://OfficeWebAppsServer:809/diskcache/DiskCache.svc, Exception:System.TimeoutException: The HTTP request to "http://OfficeWebAppsServer:809/diskcache/DiskCache.svc" has exceeded the allotted timeout of
00:00:01.9990000. The time allotted to this operation may have been a portion of a longer timeout. ---> System.Net.WebException: The request was aborted: The request was canceled. at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult result) --- End of inner exception stack trace --- Server stack trace: at
System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result) at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result) at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[]
outs, IAsyncResult result) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeEndService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage
message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData&
msgData, Int32 type) at Microsoft.Office.Web.DiskCache.IDiskCacheAsync.EndReadCacheData(IAsyncResult result) at Microsoft.Office.Web.Apps.Common.DiskCacheReaderAsync.ReadCallBack(IAsyncResult asyncResult)]
ViewStore.SetCompleted(http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…, Png, docdata.xml) - status = DocumentOutputNotFound
GlobalListLoadBalancer: Refreshing list for service WordConversionService. Reason: null hasher
GlobalListLoadBalancer: Creating consistent hasher for service WordConversionService with 50 nodes.
GlobalListLoadBalancer: Next refresh time: 06/27/2014 11:22:16
GlobalListLoadBalancer: Finished refreshing list. [Agent: WordConversionService; Time WriteLock held (ms): 3]
RequestDispatcher: dispatching request for document http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F… to http://OfficeWebAppsServer:809/wvc/Conversion.svc [attempts remaining: 12].
MachineTransfer [SourceMachine:OfficeWebAppsServer, SourceMachineVersion:15.0.4481.1508, TargetMachine:OfficeWebAppsServer, TargetMachineVersion:15.0.4481.1508, Method:ConvertingService.ProcessRequest]
Request received for document http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…, format: _v00000000-0000-0000-0000-000000000602, request id …, queued requests: 0, working requests: 0
Launching async job to add workers.
AppManager.BeginProcessRequest adding request to queue
DownloadManager: Download requested for document: http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…
MemoryDownloadCache::TryGetItem - Key:…
DownloadManager::BeginDownloadFile - Not cached. Adding to download queue
DownloadEntry::TryAddRequest - Adding request to DownloadEntry for BaseDoc:http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F… PendingRequestCount:0
DownloadManager::BeginDownloadFile - added DownloadRequest to DownloadEntry
DownloadManager: Starting download for document http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F….
RequestDispatcher: response received for document http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F… from http://OfficeWebAppsServer:809/wvc/Conversion.svc.
URL generated for WOPI GetFile Request: http://sharepoint/_vti_bin/wopi.ashx/files/…/contents?access_token=REDACTED_1040&access_token_ttl=1403904071991
Conversion response received: InProgress
WOPI Proof Data: AccessToken Hash '-2012506731' [1012 bytes], URL 'HTTP://sharepoint/_VTI_BIN/WOPI.ASHX/FILES/…/CONTENTS?ACCESS_TOKEN=REDACTED_1040&ACCESS_TOKEN_TTL=1403904071991' [1167 bytes], TimeStamp '635394648765830712' [8 bytes]
Librarian.OnConversionResponseReceived(http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…, Png, docdata.xml) - Response=InProgress, Data=NO
Librarian.SetCompleted(http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…, Png, docdata.xml) - status = InProgress
Libriarian.OnItemRetrieved: InProgress
ReadStructuredDataFromXml: [C:\Programs\OfficeWebApps\AgentManager\LanguageMapping.xml]
WOPI Proof: Using Current private key (to sign) that matches Current public key…
WOPI Proof - Successfully signed the data with the Current private key [71 ms]
WOPI Proof Keys - WS Farm State has no Old key (this is expected if recently installed)
Cannot create WOPI Proof Token - could not find valid Old proof key data
HttpRequestAsync (WOPIGetFile,WACSERVER) Starting [mode: GET, url: http://sharepoint/_vti_bin/wopi.ashx/files/…]
HttpRequestAsync::GetResponseCallback - ContentLength:20365
HttpRequestAsync::FProcessReadReturn - cbRead:20365
HttpRequestAsync::FProcessReadReturn - cbRead:0
HttpRequestAsync (WOPIGetFile,WACSERVER) Setting Completion [Time in ms: 207, Bytes Read: 20365]
HttpRequestAsync (WOPIGetFile,WACSERVER) End Called
WOPI Http Request Completed [host machine name:sharepoint, version:15.0.4420.1017, host correlation:044adcdc-0567-4727-8d82-2adb93efb266]
WopiDocument::EndRead - using Content-Length for expectedSize
WopiDocument::EndRead - Content-Length:20365 BytesRead:20365
SHA256Hash: …, Extension: .docx, FileRep: http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…
DownloadManager: Download SUCCESS for document http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…; Time spent: 397ms; Bytes Read: 20365, FileSource: WACSERVER ExpectedBytes: 20365
DownloadEntry::CompletePendingRequestsFromSingleStream - BaseDoc:http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F… TotalRequests:1
DownloadEntry::CompletePendingRequestsFromSingleStream - completing 1 DownloadRequests
MemoryDownloadCache::TryAddItem - Key:2f8b6f928ff724d93c1151fb2430b9d837f1068c30654cc5b72131df0796a8de
MemoryDownloadCache::TryAddItem - TryAdd Key:2f8b6f928ff724d93c1151fb2430b9d837f1068c30654cc5b72131df0796a8de Success:True
DownloadManager::OnFileDownloaded - Added BaseDoc:http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F… to downloadcache fAdded:True
DownloadManager::OnFileDownloaded - Tried to remove http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F… fRemoved:True
DownloadEntry::CompletePendingRequestsFromSingleStream - BaseDoc:http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F… TotalRequests:0
DownloadEntry::CompletePendingRequestsFromSingleStream - completing 0 DownloadRequests
HttpRequestAsync (WOPIGetFile,WACSERVER) SetCompletion Track start | setting trackers | WebRequest.Create() | Create() returned | _req.ContentLength 0 | setting request headers | setting user agent | setting keep alive | setting timeout callback | Start calling
StartResponseProcessing | StartResponseProcessing._req.BeginGetResponse() | BeginGetResponse() returned | StartResponseProcessing RETURNS | Start RETURNS | GetResponseCallback isSync:False | _req.EndGetResponse() | EndGetResponse() returned | RecordResponse
OK | _response.GetResponseStream() | _response.GetResponseStream() RETURNED | DoReadLoopFinished | _getStream.BeginRead() | BeginRead() -> sync:False | DoReadLoopFinished RETURNS False | GetResponseCallback DoReadLoopFinished -> FALSE | GetResponseCallback
RETURNS | ReadCallback.FProcessReadReturn() | FProcessReadReturn._getStream.EndRead() | EndRead() -> 20365 | FProcessReadReturn._responseStream.Write() | FProcessReadReturn RETURNS True | FProcessReadReturn() -> TRUE | ReadCallback.DoReadLoopFinished()
| DoReadLoopFinished | _getStream.BeginRead() | ReadCallback.sync returning | BeginRead() -> sync:True | DoReadLoopFinished.FProcessReadReturn() | FProcessReadReturn._getStream.EndRead() | EndRead() -> 0 | FProcessReadReturn._responseStream.Write() |
FProcessReadReturn RETURNS False | FProcessReadReturn() -> false | DoReadLoopFinished RETURNS true | DoReadLoopFinished() -> TRUE | SetCompletion False |
WAC Server HttpModule: Beginning request at target [/wv/ResReader.ashx?n=p1.img&WOPIsrc=http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…] in session […]
ReadCacheData Response: NoDataFound
WAC Server HttpModule: Beginning request at target [/wv/docdatahandler.ashx?WOPIsrc=http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…]
WAC Server HttpModule: Beginning request at target [/wv/ResReader.ashx?n=p1.img&WOPIsrc=http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…] in session […]
WAC Server HttpModule: Beginning request at target [/wv/docdatahandler.ashx?WOPIsrc=http%3A%2F%2Fsharepoint%2F%5Fvti%5Fbin%2Fwopi%2Eashx%2Ffiles%2F…] in session […]
What went wrong, any suggestions?