winston out, intel in

[seanmonstar]

So, this replaces our usage of winston with intel. Why?

• We get names for our loggers, so that each message has more context as to where it's coming from.
• Much of the logging moves to configuration, making it easier to test different environments.
• My super-basic-dont-rely-on benchmarks indicate 2x faster than winston.
• Uses WritableStream's internal buffering, instead of drain events: https://github.com/seanmonstar/intel/blob/master/lib/handlers/stream.js#L22
• Because ❤️

[shane-tomlinson]

@seanmonstar - travis keeps failing. :(

[seanmonstar]

I restarted the failed jobs. Looks green now.

[shane-tomlinson]

Can you update the /bin directory?

[app@ip-10-244-99-124 bin]$ ./load_gen
UNCLE: Error: Cannot find module 'winston'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/home/app/code/bin/load_gen:14:17)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
[app@ip-10-244-99-124 bin]$

[shane-tomlinson]

I did this with winston at one point too, the reason I didn't change CEF logging over is because of a comment somewhere that said "we want to see CEF logs distinct from normal logs". I like this way more.

[seanmonstar]

yea, this still has cef being used explicitly, but everything sent to cef is also passed back to us as a normal log.

[shane-tomlinson]

Cool, I like it.

[shane-tomlinson]

@seanmonstar - intel is a nice lib! All of my concerns are inline in comments already, I have not yet load tested this to see how it performs. I am seeing a substantial number of Selenium failures and am unsure whether they are caused by this PR or by something else.

[shane-tomlinson]

When running the selenium tests locally, this is the error I see:

Caused by: org.openqa.selenium.remote.ErrorHandler$UnknownServerException: Unable to locate element: {"method":"css selector","selector":"#login"}
Build info: version: '2.26.0', revision: '18041', time: '2012-11-01 19:33:38'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.9', java.version: '1.7.0_40'
Driver info: driver.version: unknown
    at <anonymous class>.<anonymous method>(file:///var/folders/x6/zd3bzsyd08v932ymg7t283rc0000gn/T/anonymous6389230538218104218webdriver-profile/extensions/fxdriver@googlecode.com/components/driver_component.js:8319)
    at <anonymous class>.<anonymous method>(file:///var/folders/x6/zd3bzsyd08v932ymg7t283rc0000gn/T/anonymous6389230538218104218webdriver-profile/extensions/fxdriver@googlecode.com/components/driver_component.js:8328)
    at <anonymous class>.<anonymous method>(file:///var/folders/x6/zd3bzsyd08v932ymg7t283rc0000gn/T/anonymous6389230538218104218webdriver-profile/extensions/fxdriver@googlecode.com/components/command_processor.js:10337)
    at <anonymous class>.<anonymous method>(file:///var/folders/x6/zd3bzsyd08v932ymg7t283rc0000gn/T/anonymous6389230538218104218webdriver-profile/extensions/fxdriver@googlecode.com/components/command_processor.js:10342)
    at <anonymous class>.<anonymous method>(file:///var/folders/x6/zd3bzsyd08v932ymg7t283rc0000gn/T/anonymous6389230538218104218webdriver-profile/extensions/fxdriver@googlecode.com/components/command_processor.js:10282)
15:22:59.550 WARN - Exception: Unable to locate element: {"method":"css selector","selector":"#login"}
Command duration or timeout: 32 milliseconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.26.0', revision: '18041', time: '2012-11-01 19:33:38'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.9', java.version: '1.7.0_40'
Driver info: driver.version: EventFiringWebDriver
Session ID: f93008e5-18ca-3249-91a8-d14810b88532
15:23:00.113 INFO - Executing: [find element: By.selector: #login] at URL: /session/1386948151112/element)
15:23:00.644 WARN - Exception thrown

[jrgm]

@shane-tomlinson actually, the ssl cert for testidp.org has expired, so a lot of selenium tests are failing at the moment.

[seanmonstar]

@shane-tomlinson you rock!

[seanmonstar]

@shane-tomlinson I rebased in fixes for many of the comments. Some I left questions back for you.

I have some work in intel@master that will allow me to remove several of the filters from lib/logging/filters and have them be in the config, removing more files and long live config!

[shane-tomlinson]

@seanmonstar - can you add this diff?

diff --git a/lib/logging/filters/statsd-increment.js b/lib/logging/filters/statsd-increment.js
index 6f6fd10..cedf569 100644
--- a/lib/logging/filters/statsd-increment.js
+++ b/lib/logging/filters/statsd-increment.js
@@ -12,8 +12,6 @@ var IncrementRegExpMatches = [
   /^wsapi\./
 ];

-uncaught_exceptions
-
 exports.test = function(msg) {
   for (var i = 0, regExp; regExp = IncrementRegExpMatches[i]; ++i) {
     if (regExp.test(msg)) return true;

[shane-tomlinson]

@seanmonstar - looking manually through the logs on a test instance, the entire configuration is printed when the processes start, meaning SMTP, database & other secret info is placed in the logs. Can this be filtered for production environments?

[shane-tomlinson]

#4045 (comment) remove more files!

[shane-tomlinson]

@seanmonstar - When attempting to run the load tests, I am seeing 503s when creating the initial users. This happened before, but I was able to complete the user creation ~50% of the time before, now I cannot at all. Quite possibly user error on my part.

[app@ip-10-105-2-239 bin]$ BROWSERID_FAKE_VERIFICATION=true ./load_gen -s https://intel-shane.personatest.org