Yesterday I wrote about Monitoring Logstash Ingest Rates with InfluxDB and Grafana, in which InfluxDB provided the data store for the ingest rate data, and Grafana the frontend. Mark Walkom reminded me on twitter that the next release of Logstash will add more functionality in this area - and that it’ll integrate back into the Elastic stack: @rmoff nice, LS 5.0 will have APIs exposing metrics too. they’ll be integrated back into Marvel/Monitoring!

Continue Reading

In this article I’m going to show you how to easily monitor the rate at which Logstash is ingesting data, as well as in future articles the rate at which Elasticsearch is indexing it. It’s a nice little touch to add to any project involving Logstash, and it’s easy to do. Logstash is powerful tool for data ingest, processing, and distribution. It originated as simply the pipe to slurp at log files and put them into Elasticsearch, but has evolved into a whole bunch more.

Continue Reading

Here’s a collection of useful articles that I’ve found over the years that give good advice on writing a good abstract, mistakes to avoid, etc: Adam Machanic - Capturing Attention: Writing Great Session Descriptions Gwen Shapira - Concrete Advice for Abstract Writers Kellyn Pot’Vin-Gorman - Abstracts, Reviews and Conferences, Oh My! Russ Unger - Conference Proposals that Don’t Suck Martin Widlake - Tips on Submitting an Abstract to Conference Bridget Kromhout - give actionable takeaways (post photo courtesy of Calum MacAulay on https://unsplash.

Continue Reading

Kibana is a tool from Elastic that makes analysis of data held in Elasticsearch really easy and very powerful. Because Elasticsearch has very loose schema that can evolve on demand it makes it very quick to get up and running with some cool visualisations and analysis on any set of data. I demonstrated this in a blog post last year, taking a CSV file and loading it into Elasticsearch via Logstash.

Continue Reading

Two vulns for OBIEE in the latest critical patch update (CPU): http://www.oracle.com/technetwork/security-advisory/cpuapr2016v3-2985753.html?elq_mid=45463&sh=91225181314122121267715271910&cmid=WWMK10067711MPP001C140 Patches is bundle patch .160419: 12.2.1: https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?parent=DOCUMENT&sourceId=2102148.1&patchId=22734181 11.1.1.9: https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?parent=DOCUMENT&sourceId=2102148.1&patchId=22393988 11.1.1.7: https://support.oracle.com/epmos/faces/ui/patch/PatchDetail.jspx?parent=DOCUMENT&sourceId=2102148.1&patchId=22225110 Note that April 2016 is the last regular patchset for 11.1.1.7, ref: https://support.oracle.com/epmos/faces/DocumentDisplay?id=2102148.1#mozTocId410847. If you’re still on it, or earlier, time to upgrade! (Photo credit: https://unsplash.com/@jenlittlebirdie)

Continue Reading

Recently added to the oracledi project over at java.net is an adaptor enabling Oracle GoldenGate (OGG) to send data to Elasticsearch. This adds a powerful alternative to [micro-]batch extract via JDBC from Oracle to Elasticsearch, which I wrote about recently over at the Elastic blog. Elasticsearch is a ‘document store’ widely used for both search and analytics. It’s something I’ve written a lot about (here and here for archives), as well as spoken about - preaching the good word, as it were, since the Elastic stack as a whole is very very good at what it does and a pleasure to work with.

Continue Reading

I’ve recently been playing around with the ELK stack (now officially known as the Elastic stack) collecting data from an IRC channel with Elastic’s Logstash, storing it in Elasticsearch and analysing it with Kibana. But, this isn’t an “ELK” post - this is a Kafka post! ELK is just some example data manipulation tooling that helps demonstrate the principles. As I wrote about last year, Apache Kafka provides a handy way to build flexible “pipelines”.

Continue Reading

On a family holiday in South Devon last week I had some good food experiences. Top of the pile was a Lardy Cake, a delicacy new to me but which I’ll be sure to be searching out again. It reminded me of an Eccles cake, but bigger and lardier! I got mine from Vinnicombes on the High Street in Sidmouth. Just down the coast from Sidmouth is a village called Beer.

Continue Reading

Timelion was released in November 2015 and with the 4.4.2 release of Kibana is available as a native visualisation once installed. It adds some powerful capabilities to Kibana as an timeseries analysis tool, using its own data manipulation language. Installing Timelion is a piece of cake: ./bin/kibana plugin -i kibana/timelion After restarting Kibana, you’ll see it as an option from the application picker There’s a bit of a learning curve with Timelion, but it’s worth it.

Continue Reading

OBIEE supports JDBC as a connection protocol, using the driver available on all installations of OBIEE, bijdbc.jar. This makes connecting to OBIEE from custom or third-party applications very easy. Once connected, you issue “Logical SQL” against the “tables” of the Presentation Layer. An example of logical SQL is: SELECT "Time"."T05 Per Name Year" saw_0 FROM "A - Sample Sales" To find more Logical SQL simply inspect your nqquery.log (obis-query.log in 12c), or Usage Tracking.

Continue Reading

OK, maybe that’s not entirely true. But my read-only client, certainly. I was perusing the Logstash input plugins recently when I noticed that there was one for IRC. Being a fan of IRC and a regular on the #obihackers channel, I thought this could be fun and yet another great example of how easy the Elastic stack is to work with. Installation is a piece of cake: wget https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/zip/elasticsearch/2.2.1/elasticsearch-2.2.1.zip wget https://download.

Continue Reading

One of the perks of my job is that I get to travel to some pretty nice places (hi, San Francisco, Bergen, Åland Islands), and get to eat some pretty good food too. If you’re looking for some techie content, then move along and go and read about Kafka, but if you enjoy food pr0n then stay put. I was working for a client in the centre of Manchester this week, staying as usual at a Premier Inn.

Continue Reading

I got this lovely failure during a fresh install of OBIEE 11.1.1.9. I emphasise that it was during the install because there’s other causes for this error on an existing system to do with corrupted credential stores etc – not the case here. The install had copied in the binaries and was in the process of building the domain. During the early stages of this where it starts configuring and restarting the AdminServer it failed, with the AdminServer.

Continue Reading

The Oracle by Example (ObE) here demonstrating how to use Goldengate to replicate transactions big data targets such as HDFS is written for the BigDataLite 4.2.1, and for me didn’t work on the current latest version, 4.4.0. The OBE (and similar Hands On Lab PDF) assume the presence of pmov.prm and pmov.properties in /u01/ogg/dirprm/. On BDL 4.4 there’s only the extract to from Oracle configuration, emov. Fortunately it’s still possible to run this setup out of the box in BDL 4.

Continue Reading

I’ve always defaulted to Slideshare for hosting slides from presentations that I’ve given, but it’s become more and more crap-infested. The UI is messy, and the UX sucks - for example, I want to download a slide deck, I most definitely 100% am not interested in “clipping” it…even if you ask me every. damn. time: Looking around it seems the other popular option is Speakerdeck. The UI is clean and simple, and I as both a user and uploader I feel like I’m there to read and share slides rather than be monetised as an eyeball on the site.

Continue Reading

#obihackers There’s a #obihackers IRC channel on freenode, where a dozen or so of us have hung out for several years now. Chat is usually OBIEE, Oracle, ODI, and general geek out. Bear in mind this is the equivalent of us hanging out in a bar; if you wanna shoot the shit with a geeky question about OBIEE go ahead, but if you’ve come to get help with your homework without even buying a round, you’ll probably get short shrift… ;-)

Continue Reading

InfluxDB is a great time series database, that’s recently been rebranded as part of the “TICK” stack, including data collectors, visualisation, and ETL/Alerting. I’ve yet to really look at the other components, but InfluxDB alone works just great with my favourite visualisation/analysis tool for time series metrics, Grafana. Getting data into InfluxDB is easy, with many tools supporting the native InfluxDB line input protocol, and those that don’t often supporting the carbon protocol (from Graphite), which InfluxDB also supports (along with others).

Continue Reading

Thanks to the power of twitter, I can look back on all the many and varied Full English breakfasts that I’ve (mostly) enjoyed: 👉🏻 https://twitter.com/search?q=rmoff%20%23fullenglish&src=typd What makes a good Full English? Good ingredients, cooked well. Nothing worse than a limp pink sausage, as it were. Sausage, standard pork or Cumberland at most. Definitely no daft apricot and guava bean with a hint of foie gras nonsense. Must be cooked right well, crispy skin, almost burnt.

Continue Reading

It struck me today when I was writing my most recent blog over at Rittman Mead that I’ve been playing with visualising OBIEE metrics for years now. Back in 2009 I wrote about using something called JManage to pull metrics out of OBIEE 10g via JMX: Still with OBIEE 10g in 2011, I was using rrdtool and some horrible-looking tcl hacking to get the metrics out through jmx : 2014 brought with it DMS and my first forays with Graphite for storing & visualising data:

Continue Reading

I will now be blogging mostly over at the venerable blog of my employer, Rittman Mead. You can see my first posting here: Web Services in BI Publisher 11g. Don’t entirely exclude this rnm1978 blog from your feeds, as I may still post more esoteric and random tidbits here.

Continue Reading