Trouble installing with ssl certificate signed by internal CA

CentOS 7.6.1810 - AWX can build fine without ssl support, and the UI is working. However trying to build the image by specifying a chained PEM cert/key combo using my internal CA is not working.

The main error is:

ChromeDriver installation failed Error with http(s) request: Error: self signed certificate in certificate chain

The full error is:

TASK [image_build : Build AWX distribution using container] ***************************************************************************************************************************************************************
fatal: [localhost → localhost]: FAILED! => {“changed”: false, “msg”: “PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm --unsafe-perm --prefix awx/ui ci --no-save awx/ui\n\n> fsevents@1.2.4 install /awx/awx/ui/node_modules/fsevents\n> node install\n\n\n> uglifyjs-webpack-plugin@0.4.6 postinstall /awx/awx/ui/node_modules/uglifyjs-webpack-plugin\n> node lib/post_install.js\n\n\n> puppeteer@1.8.0 install /awx/awx/ui/node_modules/puppeteer\n> node install.js\n\nINFO Skipping Chromium download. "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" environment variable was found.\n\n> nunjucks@3.1.3 postinstall /awx/awx/ui/node_modules/nunjucks\n> node postinstall-build.js src\n\n\n> chromedriver@2.40.0 install /awx/awx/ui/node_modules/chromedriver\n> node install.js\n\nDownloading https://chromedriver.storage.googleapis.com/2.40/chromedriver_linux64.zip\nSaving to /tmp/chromedriver/chromedriver_linux64.zip\nChromeDriver installation failed Error with http(s) request: Error: self signed certificate in certificate chain\nnpm ERR! code ELIFECYCLE\nnpm ERR! errno 1\nnpm ERR! chromedriver@2.40.0 install: node install.js\nnpm ERR! Exit status 1\nnpm ERR! \nnpm ERR! Failed at the chromedriver@2.40.0 install script.\nnpm ERR! This is probably not a problem with npm. There is likely additional logging output above.\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! /root/.npm/_logs/2019-09-19T01_29_42_681Z-debug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\nPUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm --unsafe-perm --prefix awx/ui ci --no-save awx/ui\n\n> fsevents@1.2.4 install /awx/awx/ui/node_modules/fsevents\n> node install\n\n\n> uglifyjs-webpack-plugin@0.4.6 postinstall /awx/awx/ui/node_modules/uglifyjs-webpack-plugin\n> node lib/post_install.js\n\n\n> puppeteer@1.8.0 install /awx/awx/ui/node_modules/puppeteer\n> node install.js\n\nINFO Skipping Chromium download. "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" environment variable was found.\n\n> nunjucks@3.1.3 postinstall /awx/awx/ui/node_modules/nunjucks\n> node postinstall-build.js src\n\n\n> chromedriver@2.40.0 install /awx/awx/ui/node_modules/chromedriver\n> node install.js\n\nDownloading https://chromedriver.storage.googleapis.com/2.40/chromedriver_linux64.zip\nSaving to /tmp/chromedriver/chromedriver_linux64.zip\nChromeDriver installation failed Error with http(s) request: Error: self signed certificate in certificate chain\nnpm ERR! code ELIFECYCLE\nnpm ERR! errno 1\nnpm ERR! chromedriver@2.40.0 install: node install.js\nnpm ERR! Exit status 1\nnpm ERR! \nnpm ERR! Failed at the chromedriver@2.40.0 install script.\nnpm ERR! This is probably not a problem with npm. There is likely additional logging output above.\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! /root/.npm/_logs/2019-09-19T02_41_26_233Z-debug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\nPUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm --unsafe-perm --prefix awx/ui ci --no-save awx/ui\n\n> fsevents@1.2.4 install /awx/awx/ui/node_modules/fsevents\n> node install\n\n\n> uglifyjs-webpack-plugin@0.4.6 postinstall /awx/awx/ui/node_modules/uglifyjs-webpack-plugin\n> node lib/post_install.js\n\n\n> puppeteer@1.8.0 install /awx/awx/ui/node_modules/puppeteer\n> node install.js\n\nINFO Skipping Chromium download. "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" environment variable was found.\n\n> nunjucks@3.1.3 postinstall /awx/awx/ui/node_modules/nunjucks\n> node postinstall-build.js src\n\n\n> chromedriver@2.40.0 install /awx/awx/ui/node_modules/chromedriver\n> node install.js\n\nDownloading https://chromedriver.storage.googleapis.com/2.40/chromedriver_linux64.zip\nSaving to /tmp/chromedriver/chromedriver_linux64.zip\nChromeDriver installation failed Error with http(s) request: Error: self signed certificate in certificate chain\nnpm ERR! code ELIFECYCLE\nnpm ERR! errno 1\nnpm ERR! chromedriver@2.40.0 install: node install.js\nnpm ERR! Exit status 1\nnpm ERR! \nnpm ERR! Failed at the chromedriver@2.40.0 install script.\nnpm ERR! This is probably not a problem with npm. There is likely additional logging output above.\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! /root/.npm/_logs/2019-09-19T03_05_51_039Z-debug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\n”, “status”: 2}

This is how I combined my certs and key together:

$ cat server.crt ca.crt server.key > server-ca.pem

Running openssl it seems to be fine:

$ openssl verify -CAfile ca.crt server-ca.pem
server-ca.pem: OK

In my inventory file I have made these adjustments:

#dockerhub_base=ansible

ssl_certificate=/etc/ssl/certs/server-ca.pem

Do I need to add my internal CA servers certificate to the trust store or something?

Thanks,

Realized the ‘ca_trust_dir’ option is there, and tried that but it’s still failing.

I am using a concatenated cert + key file for my server.

$ cat server.crt server.key > server.pem

I copied my CA cert file to this directory:

/etc/pki/ca-trust/source/anchors

And ran this command:

update-ca-trust

Running openssl verify still shows certificate is good.

Still getting same basic error:

Error: self signed certificate in certificate chain

Here is more of the error log with increased verbosity:

“msg”: “PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm --unsafe-perm --prefix awx/ui ci --no-save awx/ui\n\n> fsevents@1.2.4 install /awx/awx/ui/node_modules/fsevents\n> node install\n\n\n> uglifyjs-webpack-plugin@0.4.6 pos tinstall /awx/awx/ui/node_modules/uglifyjs-webpack-plugin\n> node lib/post_install.js\n\n\n> puppeteer@1.8.0 install /awx/awx/ui/node_modules/puppeteer\n> node install.js\n\nINFO Skipping Chromium download. "PUPPET EER_SKIP_CHROMIUM_DOWNLOAD" environment variable was found.\n\n> nunjucks@3.1.3 postinstall /awx/awx/ui/node_modules/nunjucks\n> node postinstall-build.js src\n\n\n> chromedriver@2.40.0 install /awx/awx/ui/node_modules /chromedriver\n> node install.js\n\nDownloading https://chromedriver.storage.googleapis.com/2.40/chromedriver_linux64.zip\nSaving to /tmp/chromedriver/chromedriver_linux64.zip\nChromeDriver installation failed Error wit h http(s) request: Error: self signed certificate in certificate chain\nnpm ERR! code ELIFECYCLE\nnpm ERR! errno 1\nnpm ERR! chromedriver@2.40.0 install: node install.js\nnpm ERR! Exit status 1\nnpm ERR! \nnpm ERR! Fa iled at the chromedriver@2.40.0 install script.\nnpm ERR! This is probably not a problem with npm. There is likely additional logging output above.\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! / root/.npm/_logs/2019-09-19T01_29_42_681Z-debug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\nPUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm --unsafe-perm --prefix awx/ui ci --no-save awx/ui\n\n> fsevents@1.2.4 install /a wx/awx/ui/node_modules/fsevents\n> node install\n\n\n> uglifyjs-webpack-plugin@0.4.6 postinstall /awx/awx/ui/node_modules/uglifyjs-webpack-plugin\n> node lib/post_install.js\n\n\n> puppeteer@1.8.0 install /awx/awx/ui/no de_modules/puppeteer\n> node install.js\n\nINFO Skipping Chromium download. "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" environment variable was found.\n\n> nunjucks@3.1.3 postinstall /awx/awx/ui/node_modules/nunjucks\n> n ode postinstall-build.js src\n\n\n> chromedriver@2.40.0 install /awx/awx/ui/node_modules/chromedriver\n> node install.js\n\nDownloading https://chromedriver.storage.googleapis.com/2.40/chromedriver_linux64.zip\nSaving t o /tmp/chromedriver/chromedriver_linux64.zip\nChromeDriver installation failed Error with http(s) request: Error: self signed certificate in certificate chain\nnpm ERR! code ELIFECYCLE\nnpm ERR! errno 1\nnpm ERR! chrome driver@2.40.0 install: node install.js\nnpm ERR! Exit status 1\nnpm ERR! \nnpm ERR! Failed at the chromedriver@2.40.0 install script.\nnpm ERR! This is probably not a problem with npm. There is likely additional loggi ng output above.\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! /root/.npm/_logs/2019-09-19T02_41_26_233Z-debug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\nPUPPETEER_SKIP_CHROMIUM_DOWNLOA D=1 npm --unsafe-perm --prefix awx/ui ci --no-save awx/ui\n\n> fsevents@1.2.4 install /awx/awx/ui/node_modules/fsevents\n> node install\n\n\n> uglifyjs-webpack-plugin@0.4.6 postinstall /awx/awx/ui/node_modules/uglifyjs- webpack-plugin\n> node lib/post_install.js\n\n\n> puppeteer@1.8.0 install /awx/awx/ui/node_modules/puppeteer\n> node install.js\n\nINFO Skipping Chromium download. "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" environment va riable was found.\n\n> nunjucks@3.1.3 postinstall /awx/awx/ui/node_modules/nunjucks\n> node postinstall-build.js src\n\n\n> chromedriver@2.40.0 install /awx/awx/ui/node_modules/chromedriver\n> node install.js\n\nDownloa ding https://chromedriver.storage.googleapis.com/2.40/chromedriver_linux64.zip\nSaving to /tmp/chromedriver/chromedriver_linux64.zip\nChromeDriver installation failed Error with http(s) request: Error: self signed certi ficate in certificate chain\nnpm ERR! code ELIFECYCLE\nnpm ERR! errno 1\nnpm ERR! chromedriver@2.40.0 install: node install.js\nnpm ERR! Exit status 1\nnpm ERR! \nnpm ERR! Failed at the chromedriver@2.40.0 install scr ipt.\nnpm ERR! This is probably not a problem with npm. There is likely additional logging output above.\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! /root/.npm/_logs/2019-09-19T03_05_51_039Z-de bug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\nPUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm --unsafe-perm --prefix awx/ui ci --no-save awx/ui\n\n> fsevents@1.2.4 install /awx/awx/ui/node_modules/fsevents\n> node ins tall\n\n\n> uglifyjs-webpack-plugin@0.4.6 postinstall /awx/awx/ui/node_modules/uglifyjs-webpack-plugin\n> node lib/post_install.js\n\n\n> puppeteer@1.8.0 install /awx/awx/ui/node_modules/puppeteer\n> node install.js\n\n INFO Skipping Chromium download. "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" environment variable was found.\n\n> nunjucks@3.1.3 postinstall /awx/awx/ui/node_modules/nunjucks\n> node postinstall-build.js src\n\n\n> chromed river@2.40.0 install /awx/awx/ui/node_modules/chromedriver\n> node install.js\n\nDownloading https://chromedriver.storage.googleapis.com/2.40/chromedriver_linux64.zip\nSaving to /tmp/chromedriver/chromedriver_linux64.zi p\nChromeDriver installation failed Error with http(s) request: Error: self signed certificate in certificate chain\nnpm ERR! code ELIFECYCLE\nnpm ERR! errno 1\nnpm ERR! chromedriver@2.40.0 install: node install.js\nn pm ERR! Exit status 1\nnpm ERR! \nnpm ERR! Failed at the chromedriver@2.40.0 install script.\nnpm ERR! This is probably not a problem with npm. There is likely additional logging output above.\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! /root/.npm/_logs/2019-09-19T03_34_13_485Z-debug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\nPUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm --unsafe-perm --prefix awx/ui ci – no-save awx/ui\nnpm ERR! Error while executing:\nnpm ERR! /usr/bin/git ls-remote -h -t ssh://git@github.com/ansible/timezone-js.git\nnpm ERR! \nnpm ERR! fatal: Unable to read current working directory: No such file or d irectory\nnpm ERR! \nnpm ERR! exited with error code: 128\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! /root/.npm/_logs/2019-09-19T03_49_32_829Z-debug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\nPUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm --unsafe-perm --prefix awx/ui ci --no-save awx/ui\n\n> fsevents@1.2.4 install /awx/awx/ui/node_modules/fsevents\n> node install\n\n\n> uglifyjs-webpack-plugin@0.4.6 posti nstall /awx/awx/ui/node_modules/uglifyjs-webpack-plugin\n> node lib/post_install.js\n\n\n> puppeteer@1.8.0 install /awx/awx/ui/node_modules/puppeteer\n> node install.js\n\nINFO Skipping Chromium download. "PUPPETEE R_SKIP_CHROMIUM_DOWNLOAD" environment variable was found.\n\n> nunjucks@3.1.3 postinstall /awx/awx/ui/node_modules/nunjucks\n> node postinstall-build.js src\n\n\n> chromedriver@2.40.0 install /awx/awx/ui/node_modules/c hromedriver\n> node install.js\n\nDownloading https://chromedriver.storage.googleapis.com/2.40/chromedriver_linux64.zip\nSaving to /tmp/chromedriver/chromedriver_linux64.zip\nChromeDriver installation failed Error with http(s) request: Error: self signed certificate in certificate chain\nnpm ERR! code ELIFECYCLE\nnpm ERR! errno 1\nnpm ERR! chromedriver@2.40.0 install: node install.js\nnpm ERR! Exit status 1\nnpm ERR! \nnpm ERR! Fail ed at the chromedriver@2.40.0 install script.\nnpm ERR! This is probably not a problem with npm. There is likely additional logging output above.\n\nnpm ERR! A complete log of this run can be found in:\nnpm ERR! /ro ot/.npm/_logs/2019-09-19T04_07_34_078Z-debug.log\nmake: *** [awx/ui/.release_deps_built] Error 1\n”,
“status”: 2
}

Can someone help me understand why its still failing? Are there other openssl commands I can run, or other log files I can check? Thank you!

I opened a bug report for this:

https://github.com/ansible/awx/issues/4786