As you may have noticed, I’m now back at ASG in the role of Corporate Technologist, helping guide the strategic direction of our emerging technology offerings. One of my first experiences back at ASG was attending the Elastic Engineer 1 class in Denver, with Dan Park and Tom Whitcomb. It was quite an eye-opener to the world of everything search. Elastic is clearly a leader in this space as acknowledged by the full class and depth of clients attending the training (DOD, Dept of Interior, Dish, CVS Health, Salesforce). I wanted to share a few takeaways from my experience.
My first takeaway:
While the Elastic Stack (ES) has many capabilities, deploying AVA – Advanced Vector Automation alongside ES adds numerous valuable enhancements, as well as ease-of-deployment. Elastic is a cluster-based system with master, data, ingest and coordinating nodes. Each node maintains at least 3 config files including its own elasticsearch.yml and Java configs. And you guessed it, you must touch each one of these files across your cluster. In class, we built a 3-node cluster and it quickly got burdensome and error-prone just managing 3 nodes. Most Elastic clusters are larger than 25 nodes, but thankfully AVA manages and configs your nodes for you. And with ASG recently getting a big client win on AVA, I now can see many more future engagements using AVA as a solution to quickly get clients up and running with ES.
Another reason why I would won’t attempt Elastic without using AVA is log collection. Even though we didn’t cover Beats or LogStash, many people (including Elastic employees) bemoaned of the issues getting your log data correctly into ES. Again, AVA provides proven methods to capture logs, event and config data across the data center.
My second takeaway:
The power of search is the new IT application. Although it might seem trivial to “grep” through thousands of documents, logs or event streams, ES does it in milliseconds. The power of this capability is mind-expanding for a typical IT organization. Of course, this type of search has been the part of the underpinning for social media and e-commerce sites, most IT organizations have not yet embraced its benefits. And when you add the power of visualization to search, operational dashboards have never looked so cool (and accurate).
My third takeaway:
If you are writing an application that needs search, ES is probably all you need. However, if you are intending to use it in your data center, then you need some sort of automation and/or machine learning capabilities. Again, AVA to the rescue. Not only do you get a quick ES implementation, but you also get automation that can do many useful things, like automatically re-provision servers, fix config drift issues, change application traffic policies for load reasons, automatically block IPs for security purposes… And over time, AVA learns more and more about your data center environment from all the historical data you give it.
If you’re interested in learning more about AVA, you can sign up for a free data center AIOps workshop – AIOps Strategy and IT Transformation.