This may be a Microsoft Tech Support call, but, giving it a shot here first.
Env: SharePoint 2013 on premise, 2 server farm (SP and DB). Not fully patched and updated (it's on the list to do). 5 Web Apps (including central admin and mysites). Multiple sites and site collections, lists, libraries.
About a week ago (from 9/10/19) I started getting calls about workflows failing "Failed on Start" "Workflow was canceled by System Account." Found lots of talk about workflows failing just like mine. Found this from developer support:
Thought okay, I'll check the hidden library. The link on how to nav to the library (kb/2273439) returns 404, eventually found this article about the hidden workflow library: https://blogs.technet.microsoft.com/sposupport/2017/10/13/troubleshooting-the-sharepoint-2013-workflow-hidden-library/
Give it a go, looking for the GUID of wfsvc and the url returns "List 'wfsvc' does not exist at the site..." Thinking this can't be good, I test against another of my web apps and it returns the expected information. Uh oh. So I get on PowerShell, grab the web (SP-Web), grab the lists, search for the list named 'wfsvc' and it doesn't show in the web app with the failing workflows. My PowerShell does show the list on the web app where workflows are working.
Poke around a bit more and find mention of running SQL on the dbs to do something with wfsvc - alarm bells go off - not messing with the tables via SQL. But then I'm thinking I can just query and see if it's in there, so I do a quick Select on dbo.AllLists looking for tp_title = 'wfsvc'. I find it in the db for the web app that is working fine, but, the web app where the workflows aren't launching there's no joy on the query, no rows returned. So now I know, close SQL and sigh in relief that I didn't trash the dbs.
But, now what? wfsvc seems to be gone from the web app. I tried turning off all the Workflow stuff in Site Collection Features then turning it back on, thinking that might recreate the list. I ran PSCONFIGUI on the server and let it run through what it wanted to run through (a repair cycle as I recall) and that ran through to completion. Maybe I didn't turn off enough stuff - Workflows, Three-state workflow, SharePoint 2007 Workflows (were never turned on!), Publishing Approval Workflow (wasn't turned on), Disposition Approval Workflow. Is there something in Central Admin to toggle? I don't see anything there except the Workflow Service Application, I don't see any controls under Service Apps for it, and I don't want to endanger the other web apps.
Thanks all,
Steven