Scenario:
We have few agents scheduled which runs fine everyday. Few of the agents delivery content is format is Excel 2007 and for few is CSV and PDF. All of the sudden most of the agents did not run as per the schedule and failed after many trails. when we try to run the agents manually from Presentation Services we are getting the same error.
When we go back and check the agents we found that only the agents whose delivery content is Excel2007 got failed but the PDF and CSV format agents ran successfully. When we were checking the log file we got following error.
Agent Log File:
[2015-09-11T12:00:16.000-05:00] [OracleBISchedulerComponent] [ERROR:1] [] [] [ecid: 0009Z_lvvMs5yW85njT4iZ0001Rn0007nk,0:1:2] [tid: b] [nQSError: 77006] Oracle BI Presentation Server Error: A fatal error occurred while processing the request. The server responded with: Path not found (/shared/xxxxx/xxxx/xxxx) [[
Error Codes: U9KP7Q94
.
Error Codes: AGEGTYVF
]]
[2015-09-11T12:00:16.000-05:00] [OracleBISchedulerComponent] [ERROR:1] [] [] [ecid: 0009Z_lvvMs5yW85njT4iZ0001Rn0007nk,0:1:2] [tid: b] AgentID: xxxxxx/xxxxxx/xxxxxx/xxxxx[[
...Trying main Agent loop again.
]]
[2015-09-11T12:00:16.000-05:00] [OracleBISchedulerComponent] [ERROR:1] [] [] [ecid: 0009Z_lvvMs5yW85njT4iZ0001Rn0007nk,0:1:2] [tid: b] ... Sleeping for 2 seconds.
[2015-09-11T12:00:18.000-05:00] [OracleBISchedulerComponent] [ERROR:1] [] [] [ecid: 0009Z_lvvMs5yW85njT4iZ0001Rn0007nk,0:1:2] [tid: b] [nQSError: 77006] Oracle BI Presentation Server Error: A fatal error occurred while processing the request. The server responded with: Path not found (/shared/xxxx/xxxxx/xxxxxx) [[
Error Codes: U9KP7Q94
.
Error Codes: AGEGTYVF
]]
[2015-09-11T12:00:18.000-05:00] [OracleBISchedulerComponent] [ERROR:1] [] [] [ecid: 0009Z_lvvMs5yW85njT4iZ0001Rn0007nk,0:1:2] [tid: b] AgentID: /shared/xxxxx/xxxxx/xxxx [[
...Trying main Agent loop again.
]]
[2015-09-11T12:00:18.000-05:00] [OracleBISchedulerComponent] [ERROR:1] [] [] [ecid: 0009Z_lvvMs5yW85njT4iZ0001Rn0007nk,0:1:2] [tid: b] ... Sleeping for 8 seconds.
[2015-09-11T12:00:26.000-05:00] [OracleBISchedulerComponent] [ERROR:1] [] [] [ecid: 0009Z_lvvMs5yW85njT4iZ0001Rn0007nk,0:1:2] [tid: b] [nQSError: 77006] Oracle BI Presentation Server Error: A fatal error occurred while processing the request. The server responded with: Path not found (/shared/xxxx/xxxx/xxxxx) [[
Error Codes: U9KP7Q94
.
Error Codes: AGEGTYVF
]]
[2015-09-11T12:00:26.000-05:00] [OracleBISchedulerComponent] [ERROR:1] [] [] [ecid: 0009Z_lvvMs5yW85njT4iZ0001Rn0007nk,0:1:2] [tid: b] AgentID: /shared/xxxx/xxxxx/xxxxx[[
Exceeded number of request retries for method RuniBot.
]]
Scheduler Log File (Obisch1.log):
processing the request. The server responded with: Invalid Handle Of 'saw::RPCClientEndPoint*' Exception
Error Codes: Q4NU7XSN
Location: saw.subsystem.portal.pdf, saw.delivers.rpc.getDeviceContent, saw.rpc.server.responder, saw.rpc.server, saw.rpc.server.handleConnection, saw.rpc.server.dispatch, saw.threadpool.socketrpcserver, saw.threads
.
[Util Logger] Exception occurred:
Severity:1
Type:const saw::Exception*
File:project/webcomm/rpc.cpp
Line:479
Message:A fatal error occurred while processing the request. The server responded with: Invalid Handle Of 'saw::RPCClientEndPoint*' Exception
Error Codes: Q4NU7XSN
Location: saw.subsystem.portal.pdf, saw.delivers.rpc.getDeviceContent, saw.rpc.server.responder, saw.rpc.server, saw.rpc.server.handleConnection, saw.rpc.server.dispatch, saw.threadpool.socketrpcserver, saw.threads
.
[Util Logger] Exception occurred:
Severity:1
Type:const saw::Exception*
File:project/webcomm/rpc.cpp
Line:479
Message:A fatal error occurred while processing the request. The server responded with: Path not found (/shared/xxxx/xxxxxx)
Error Codes: U9KP7Q94
Error Codes: Q4NU7XSN
Location: saw.subsystem.portal.pdf, saw.delivers.rpc.getDeviceContent, saw.rpc.server.responder, saw.rpc.server, saw.rpc.server.handleConnection, saw.rpc.server.dispatch, saw.threadpool.socketrpcserver, saw.threads
.
[Util Logger] Exception occurred:
Severity:1
Type:const saw::Exception*
File:project/webcomm/rpc.cpp
Line:479
Message:A fatal error occurred while processing the request. The server responded with: Invalid Handle Of 'saw::RPCClientEndPoint*' Exception
Error Codes: Q4NU7XSN
Location: saw.subsystem.portal.pdf, saw.delivers.rpc.getDeviceContent, saw.rpc.server.responder, saw.rpc.server, saw.rpc.server.handleConnection, saw.rpc.server.dispatch, saw.threadpool.socketrpcserver, saw.threads
.
[Util Logger] Exception occurred:
Severity:1
Type:const saw::Exception*
File:project/webcomm/rpc.cpp
Line:479
Message:A fatal error occurred while processing the request. The server responded with: Path not found (/shared/xxxx/xxxxxx)
Error Codes: U9KP7Q94
Root Cause for the Issue:
We have back tracked the changes that we have done recently and found that the permissions on the WebCatalog got changed. The folder where the agents are placed and the Admin User(In our case its Weblogic) Folder from where the agents will start running had only Read Only access.Resolution:
We have reset the permissions for weblogic user to Full Control using Catalog Manager (for all its folders and sub folders). It worked all the agents ran successfully.
Note: No need to Restart Obiee OPMN Services after resetting the permissions.