northben's blog

Normally, I prefer to send CSV or JSON data to Splunk. But sometimes XML can't be avoided. I recently needed to ingest an XML file, and through judicious use of 'MUST_BREAK_AFTER' and 'BREAK_ONLY_BEFORE' in props.conf, I was able to extract the events from the XML file that looked like this:

In this short tutorial I will show you how to create a dashboard table with dynamic columns. When the user selects the radio button toggles, the search does NOT run again -- only the display is updated. Demo: https://youtu.be/l-p83je4RgQ

I am including the full source code to the dashboard with this post for your review. But here's the secret sauce:

I've been having trouble indexing CSV files. In particular, CSV files from Tripwire. I'll show you the format and how I was able to index the files in Splunk

Node Name,Node Type,Policy,Parent Test Group,Test Name,Description,Element,Result Time,Result State,Actual Value
"192.168.1.1",Linux Server,"My Policy Name","My Test Group","My Test Name","My Test Description","Some Element",10/25/15 2:02 AM,passed,"ELEMENT=foo"

Here's my Props.conf stanza:

I needed to prevent a webpage from closing automatically. Turns out this is very easy to do with JavaScript!

I've been working with Splunk Enterprise a lot lately (and it's very powerful and easy to use!). In many situations, it is useful to show some metric compared to the same metric a month ago (or some other time period).

One way to accomplish this is with the community-supported Splunk app, Timewrap. I couldn't get Timewrap to output the data as I wanted, so instead here's the approach that I used.

I needed to send an ICAP request to a Symantec Antivirus server. Because ICAP is HTTP-like, but not quite HTTP, I could not use the wonderful requests library. So, here's what I did instead:

Let's say you need to update lots of keys in Amazon S3. If you have many objects in your S3 bucket, this can be quite slow. Of course, as a Python developer, you're using the nifty boto library. We can make update all of your keys much, much faster using multiple threads!

Background:

I was setting up a development copy of a client website on my computer the other day, and after I had cloned the repo, set up the database, and pip installed Django and all of the other python packages needed for the website, I faced a strange Exception coming from deep inside of one of my dependencies. After an embarrasingly long period of troubleshooting, I realized that when I specified my python dependencies, I neglected to include the version requirements for each package.

Problem:

I wanted to provide a super simple example of using Ned Batchelder's coverage.py for testing my Python unittest coverage. The thing that tripped me up at first is that I needed to call coverage.py in such a way to exercise Python's unittest framework. For example:

coverage run -m unittest discover;
coverage report -m;

Mocking dates is a well-known PITA with Python. But here's a quick explanation of how I worked around this deficiency.

Step 1: Add the date as a property to the production class. I had to refactor my code, and I suspect you will too. Before mocking this date, I was calling date.today() from the build_widget method.

from datetime import date

class WidgetWorker(object):
    date = date.today()
    def build_widget(self):
        return {'date_created': self.date}