Win_ping Module | failed to run exec_wrapper | URGENT

[Piyush_Bansal]

Hello,

I am running win_ping for windows 2008R2 64bit server from ansible 2.8.2 & python version = 2.7.5. Below are connection parameters:
winrm
ssl
5986
ignore certificates
Admin user
Credentials are working fine and no issues.

While running this task I am getting following error. Never have faced such error before.
Please be so kind to suggest on this.

TASK [win_ping] *********************************************************************************************************************************************
Using module file /usr/lib/python2.7/site-packages/ansible/modules/windows/win_ping.ps1
Pipelining is enabled.
<a.b.c.d> ESTABLISH WINRM CONNECTION FOR USER: adm on PORT 5986 TO a.b.c.d
EXEC (via pipeline wrapper)
The full traceback is:
Failed to compile C# code:
error CS0013: Unexpected error writing metadata to file ‘c:\Users\Adm\AppData\Local\Temp\xyz.dll’ – ‘��"’
At line:279 char:13

• throw [InvalidOperationException]$msg
• CategoryInfo : OperationStopped: ( , InvalidOperationException
• FullyQualifiedErrorId : Failed to compile C# code:
  error CS0013: Unexpected error writing metadata to file ‘c:\Users\Adm\AppData\Local\Temp\xyz.dll’ – ‘��"’

ScriptStackTrace:
at Add-CSharpType, : line 279
at , : line 19
at , : line 137
at , : line 7

fatal: [a.b.c.d]: FAILED! => changed=false
msg: |-
internal error: failed to run exec_wrapper action module_powershell_wrapper: Failed to compile C# code:
error CS0013: Unexpected error writing metadata to file ‘c:\Users\Adm\AppData\Local\Temp\xyz.dll’ – ‘"’

Let me know if you need further details…!!
I have changed the dll files,IP address, user etc details for obvious security reasons.

Thanks,
Piyush

[j.r.hawkesworth]

I have not seen this error before, but what permissions are on
c:\Users\Adm\AppData\Local\Temp\ directory?

Do you get any more information if you run ansible with -vvvvvv option?

[jborean (Jordan Borean)]

We use C# code in our code for a lot of things, in this case it’s the Ansible wrapper we use to abstract the communication to and from the controller. Part of that process is to compile the C# code to a dll and load that in PowerShell. This all happens in the background and is part of PowerShell, similar to how the Add-Type cmdlet works.

If it’s failing to write to a file in the temp directory then it could be an issue with;

• Not having permissions to the dir
• Antivirus blocking the process
  I don’t think it’s the permissions issue as that’s the temp dir for the current user but it’s probably still a good idea to check. As for the anti-virus, it could be seeing a process writing a dll as something suspicious and just block it from doing so. Unfortunately this is how Ansible compiles the C# code for use in it’s modules and we can’t really do too much about that. I would check to see if there are any AV logs to see if it logged a reason why it blocked access.

[Piyush_Bansal]

Hello Jordan and Hawkesworth,

Many Thanks for your email…!!

I disabled all services of Anti-Virus software on the server and tried win_ping but still the same issue.
Tried with unrestricted, remotesigned and bypass execution policy but still win_ping does not work.

Local user account which I am using to connect to windows server definitely has full access so, full permissions every where on this server.

Please note that it is not giving any permission not found on any file rather it says file not found…!!!

Below is the traceback with -vvvvv verbosity:

TASK [win_ping] *********************************************************************************************************************************************
Using module file /usr/lib/python2.7/site-packages/ansible/modules/windows/win_ping.ps1
Pipelining is enabled.
<a.b.c.d> ESTABLISH WINRM CONNECTION FOR USER: administrator on PORT 5986 TO a.b.c.d
<a.b.c.d> WINRM CONNECT: transport=ssl endpoint=https://a.b.c.d:5986/wsman
<a.b.c.d> WINRM OPEN SHELL: 6557236D-D283-4BAC-AFF8-E5FFC26B3265
EXEC (via pipeline wrapper)
<a.b.c.d> WINRM EXEC ‘PowerShell’ [‘-NoProfile’, ‘-NonInteractive’, ‘-ExecutionPolicy’, ‘Unrestricted’, ‘-EncodedCommand’, u’UAAGUAZABDAG8AbQBtAGEAbgBkACAASgBnAEIAagBBAEcAZwBBAFkAdwBCAHcAQQBDADQAQQBZAHcAQgB2AEEAAQgAwAEEARwBVAE******ALPHANUMERICCHARSREMOVEDFORBEVERTY8EASwBBAEEAawBBAEgATQBBAGMAQQBCAHMAQQBHAGsAQQBkAEEAQgBmAEEASABBAEEAWQBRAEIAeQBBAEgAUQBBAGMAdwBCAGIAQQBEAEEAQQBYAFEAQQBwAEEAQQBvAEEASgBnAEEAawBBAEcAVQBBAGUAQQBCAGwAQQBHAE0AQQBYAHcAQgAzAEEASABJAEEAWQBRAEIAdwBBAEgAQQBBAFoAUQBCAHkAQQBBAD0APQA=‘]
<a.b.c.d> WINRM RESULT u’<Response code 1, out "{“exception”:“Failed”, err “”>’
<a.b.c.d> WINRM STDOUT {“exception”:“Failed to compile C# code: \r\nerror CS0010: Unexpected fatal error – \u0027n\u0027\r\nAt line:279 char:13\r\n+ throw [InvalidOperationException]$msg\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : OperationStopped: ( , InvalidOperationException\r\n + FullyQualifiedErrorId : Failed to compile C# code: \r\nerror CS0010: Unexpected fatal error – \u0027n\u0027\r\n\r\nScriptStackTrace:\r\nat Add-CSharpType, \u003cNo file\u003e: line 279\r\nat \u003cScriptBlock\u003e, \u003cNo file\u003e: line 19\r\nat \u003cScriptBlock\u003e\u003cEnd\u003e, \u003cNo file\u003e: line 137\r\nat \u003cScriptBlock\u003e, \u003cNo file\u003e: line 7\r\n”,“msg”:“internal error: failed to run exec_wrapper action module_powershell_wrapper: Failed to compile C# code: \r\nerror CS0010: Unexpected fatal error – \u0027n\u0027”,“failed”:true}

<a.b.c.d> WINRM STDERR
<a.b.c.d> WINRM CLOSE SHELL: 6558656D-D283-4BAC-AJJ8-E5FFC26B3612
The full traceback is:
Failed to compile C# code:
error CS0010: Unexpected fatal error – ‘n’
At line:279 char:13

• throw [InvalidOperationException]$msg
• CategoryInfo : OperationStopped: ( , InvalidOperationException
• FullyQualifiedErrorId : Failed to compile C# code:
  error CS0010: Unexpected fatal error – ‘n’

ScriptStackTrace:
at Add-CSharpType, : line 279
at , : line 19
at , : line 137
at , : line 7

fatal: [a.b.c.d]: FAILED! => changed=false
msg: |-
internal error: failed to run exec_wrapper action module_powershell_wrapper: Failed to compile C# code:
error CS0010: Unexpected fatal error – ‘n’

Thanks,
Piyush

[Piyush_Bansal]

Hello,

I allowed unencrypted on winrm client and winrm service with unrestricted execution-policy.
Below is the detailed traceback for win_ping:

TASK [win_ping] *********************************************************************************************************************************************
Using module file /usr/lib/python2.7/site-packages/ansible/modules/windows/win_ping.ps1
Pipelining is enabled.
<a.b.c.d> ESTABLISH WINRM CONNECTION FOR USER: ADM on PORT 5986 TO a.b.c.d
<a.b.c.d> WINRM CONNECT: transport=ssl endpoint=https://a.b.c.d:5986/wsman
<a.b.c.d> WINRM OPEN SHELL: 79601CF5-4113-4573-9E10-BC60C19BCD21
EXEC (via pipeline wrapper)
<a.b.c.d> WINRM EXEC ‘PowerShell’ [‘-NoProfile’, ‘-NonInteractive’, ‘-ExecutionPolicy’, ‘Unrestricted’, ‘-EncodedCommand’, u’UABvAHcAZQByAFMAaABlAGwAbAAgACsadsfasdfasdfasAcgBhAGMAdABpAHYAZQAgAC0ARQB4AGUAYwB1AHQAaQBvAG4AUABvAGwAaQBjAHkAIABVAG4AcgBlAHMAdAByAGkAYwB0AGUAZAAgAC0ARQBuAGMAbwBkAGUAZABDALPHANUMERICCHARREMOVEDFORBEVERTY************QQBLAEEAQwBRAEEAWgBRAEIANABBAEcAVQBBAFkAdwBCAGYAQQBIAGMAQQBjAGcAQgBoAEEASABBAEEAYwBBAEIAbABBAEgASQBBAEkAQQBBADkAQQBDAEEAQQBXAHcAQgBUAEEARwBNAEEAYwBnAEIAcABBAEgAQQBBAGQAQQBCAEMAQQBHAHcAQQBiAHcAQgBqAEEARwBzAEEAWABRAEEANgBBAEQAbwBBAFEAdwBCAHkAQQBHAFUAQQBZAFEAQgAwAEEARwBVAEEASwBBAEEAawBBAEgATQBBAGMAQQBCAHMAQQBHAGsAQQBkAEEAQgBmAEEASABBAEEAWQBRAEIAeQBBAEgAUQBBAGMAdwBCAGIAQQBEAEEAQQBYAFEAQQBwAEEAQQBvAEEASgBnAEEAawBBAEcAVQBBAGUAQQBCAGwAQQBHAE0AQQBYAHcAQgAzAEEASABJAEEAWQBRAEIAdwBBAEgAQQBBAFoAUQBCAHkAQQBBAD0APQA=‘]
<a.b.c.d> WINRM RESULT u’<Response code 1, out "{“exception”:“Except”, err “”>’
<a.b.c.d> WINRM STDOUT {“exception”:“Exception calling "CompileAssemblyFromDom" with "2" argument(s): "Could not find file \u0027C:\Users\ADM\AppData\Local\Temp\nzq3veem.dll\u0027."\r\nAt line:19 char:5\r\n+ Add-CSharpType -References $csharp_utils -TempPath $new_tmp -IncludeDebugInf …\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: ( [Add-CSharpType], MethodInvocationException\r\n + FullyQualifiedErrorId : FileNotFoundException,Add-CSharpType\r\n\r\nScriptStackTrace:\r\nat Add-CSharpType, \u003cNo file\u003e: line 273\r\nat \u003cScriptBlock\u003e, \u003cNo file\u003e: line 19\r\nat \u003cScriptBlock\u003e\u003cEnd\u003e, \u003cNo file\u003e: line 137\r\nat \u003cScriptBlock\u003e, \u003cNo file\u003e: line 7\r\n\r\nSystem.Management.Automation.MethodInvocationException: Exception calling "CompileAssemblyFromDom" with "2" argument(s): "Could not find file \u0027C:\Users\ADM\AppData\Local\Temp\nzq3veem.dll\u0027." —\u003e System.IO.FileNotFoundException: Could not find file \u0027C:\Users\ADM\AppData\Local\Temp\nzq3veem.dll\u0027.\r\n at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)\r\n at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)\r\n at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)\r\n at System.IO.File.InternalReadAllBytes(String path, Boolean checkHost)\r\n at Microsoft.CSharp.CSharpCodeGenerator.FromFileBatch(CompilerParameters options, String fileNames)\r\n at Microsoft.CSharp.CSharpCodeGenerator.FromDomBatch(CompilerParameters options, CodeCompileUnit ea)\r\n at Microsoft.CSharp.CSharpCodeGenerator.System.CodeDom.Compiler.ICodeCompiler.CompileAssemblyFromDomBatch(CompilerParameters options, CodeCompileUnit ea)\r\n at CallSite.Target(Closure , CallSite , Object , Object , Object )\r\n — End of inner exception stack trace —\r\n at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)\r\n at System.Management.Automation.Interpreter.ActionCallInstruction2.Run(InterpretedFrame frame)\r\n at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)\r\n at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)\r\n at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame)\r\n at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0)\r\n at System.Management.Automation.PSScriptCmdlet.RunClause(Action1 clause, Object dollarUnderbar, Object inputToProcess)\r\n at System.Management.Automation.CommandProcessorBase.Complete()”,“msg”:“internal error: failed to run exec_wrapper action module_powershell_wrapper: Exception calling "CompileAssemblyFromDom" with "2" argument(s): "Could not find file \u0027C:\Users\ADM\AppData\Local\Temp\nzq3veem.dll\u0027."”,“failed”:true}

<a.b.c.d> WINRM STDERR
<a.b.c.d> WINRM CLOSE SHELL: 79601CF5-4113-4573-9E10-BC60C19BCD21
The full traceback is:
Exception calling “CompileAssemblyFromDom” with “2” argument(s): “Could not find file ‘C:\Users\ADM\AppData\Local\Temp\nzq3veem.dll’.”
At line:19 char:5

• Add-CSharpType -References $csharp_utils -TempPath $new_tmp -IncludeDebugInf …
• CategoryInfo : NotSpecified: ( [Add-CSharpType], MethodInvocationException
• FullyQualifiedErrorId : FileNotFoundException,Add-CSharpType

ScriptStackTrace:
at Add-CSharpType, : line 273
at , : line 19
at , : line 137
at , : line 7

System.Management.Automation.MethodInvocationException: Exception calling “CompileAssemblyFromDom” with “2” argument(s): “Could not find file ‘C:\Users\ADM\AppData\Local\Temp\nzq3veem.dll’.” —> System.IO.FileNotFoundException: Could not find file ‘C:\Users\ADM\AppData\Local\Temp\nzq3veem.dll’.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.File.InternalReadAllBytes(String path, Boolean checkHost)
at Microsoft.CSharp.CSharpCodeGenerator.FromFileBatch(CompilerParameters options, String fileNames)
at Microsoft.CSharp.CSharpCodeGenerator.FromDomBatch(CompilerParameters options, CodeCompileUnit ea)
at Microsoft.CSharp.CSharpCodeGenerator.System.CodeDom.Compiler.ICodeCompiler.CompileAssemblyFromDomBatch(CompilerParameters options, CodeCompileUnit ea)
at CallSite.Target(Closure , CallSite , Object , Object , Object )
— End of inner exception stack trace —
at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
at System.Management.Automation.Interpreter.ActionCallInstruction2.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0) at System.Management.Automation.PSScriptCmdlet.RunClause(Action1 clause, Object dollarUnderbar, Object inputToProcess)
at System.Management.Automation.CommandProcessorBase.Complete()
fatal: [a.b.c.d]: FAILED! => changed=false
msg: ‘internal error: failed to run exec_wrapper action module_powershell_wrapper: Exception calling “CompileAssemblyFromDom” with “2” argument(s): “Could not find file ‘‘C:\Users\ADM\AppData\Local\Temp\nzq3veem.dll’’.”’

[jborean (Jordan Borean)]

I completely spaces that you are running against Server 2008 R2. There’s a bug with PowerShell v3 and remoting around out of memory issues which commonly appears like stuff you are seeing here. Make sure you have met the pre-reqs and either

• Update PowerShell to v5 preferably
• Install the Windows hotfix for v3 [1]
  The hotfix is hard to get but that link should have enough details to get you started. I would highly recommend you upgrade to PowerShell v5 anyway as that contains some nice logging features you can take advantage of.

[1] - https://docs.ansible.com/ansible/latest/user_guide/windows_setup.html#winrm-memory-hotfix

Thanks

Jordan