Developing Spotfire Skills

A Spotfire user reached out to me on LinkedIn last week asking about developing Spotfire skills. He wanted to know what skills to focus on and how to develop them. I have LOTS of ideas and thoughts on this. Therefore, this week’s post looks at building Spotfire skills based on roles in the oil and gas industry.

I have personally built and delivered four different Spotfire courses at two companies to hundreds of users. I also built the first version of Ruths.ai’s online Essentials training. Building and delivering all that content has given me a comprehensive understanding of the application and the people using it.

To begin, I want to talk about roles and responsibilities because not all Spotfire users need the same skillsets. How I define roles and responsibilities relates to what I’ve seen working in oil and gas over the last 15 years. From there, I’ll break skills into Beginner, Intermediate, Advanced, and Master skill sets. Lastly, I’ll explain what’s in each skill set and then how to develop them. Note, Spotfire admin skills are out of scope.

In a week or two, I’ll explain where roles and skills are going because if there is one thing that’s true in tech — it’s always changing. And I can promise you, the future looks very different from the past.

Roles & Responsibilities

The people that I work with on a day to day basis using Spotfire are these folks shown below. Each of these roles is expected to work with Spotfire in some way shape or form, and we’ll look at them one at a time.

• Techs
• Analysts
• Engineers
• Scientists
• Managers
• Analytics Professionals

Techs & Analysts

The tech role is a specific position that works with engineers and managers in functional disciplines like geo, drilling, and completions. They have plenty of domain knowledge but not necessarily the engineering degree. Three to five years ago, this was where many companies tried to put people with analytics skills (like me) before stand-alone analytics divisions became common.

Now, analysts don’t work in the same functional disciplines as techs. Rather, you’ll find analysts in finance, supply chain, and HR.

Techs and analysts can have any level of Spotfire skill, depending on what their particular manager expects of them and their development. In some cases, techs/analysts develop intermediate to advanced Spotfire skills because managers expect them to build sophisticated projects. I’ve also seen techs and analysts use only the web player to QAQC data and/or feed data thru a process developed by an analytics professional. In these instances, they only need beginner skills.

Engineers

Most engineers working with Spotfire have intermediate to advanced skills (or aspire to advanced skills). Granted, they may not have advanced skills in all coding areas. They might specialize in one or two, but they know the application well. They also know how to work with data — profiling, preparation, joining, etc. Now, I don’t actually expect this trend to continue, but you’ll have to wait for part 2. Then, I’ll explain.

Scientists

Scientists seem to use Spotfire a bit differently from the other roles mentioned so far. They mainly use Spotfire for the visualization aspect because they use other applications like Petra or Kingdom to do their work. They will have a mix of beginner and intermediate skills.

Of course, there are always exceptions. Three of our scientists collaborated with Ruths.ai to write a paper on a new methodology merging seismic, geologic, and engineering data to predict completion performance, which was done with Spotfire. You can find an abscract of that here.

Managers

Managers primarily work with the web player to consume content created by techs, analysts, or engineers. They may consume content in the desktop app, as some managers like to be able to make tweaks here and there. The web player is a good choice in cost-conscious environments. Thus, they only need to have beginner skills. It does help if they are aware of what can be done with intermediate and advanced skills to communicate requirements.

Analytics Professionals

The term Analytics Professional refers to folks like myself who are part of an analytics team, although the term can also apply to folks whose roles revolve around creating Spotfire projects, such as the Report Developer. These folks must have Advanced skills and might also be working towards Master skills.

I intentionally didn’t name the Data Scientist as a role, but they may fall in this group. I want to mention them because I know readers will ask — what about data scientists??? However, I haven’t seen a lot of true data science done in Spotfire. At the Gartner conference this week, one speaker noted (paraphrasing) — the fastest way to lose your data scientist is to tell them they can use one and only one specific tool. Furthermore, with the acquisitions of Statistica and Alpine Data Science, TIBCO performs data science in the TIBCO Data Science product rather than in Spotfire, which I think is a good move.

So with that explanation of roles and responsibilities, let’s talk about skills.

Beginner Skills

• Importing data from different data sources
• Basics of data wrangling
• Using the panels
• Marking
• Creating visualizations
• Basic calculations
• Using the web player and its controls

Intermediate Skills

• Advanced data wrangling
• Data limiting (including details visualizations)
• Using the Spotfire expression language to create calculated columns
• Using lines and curves
• Working with Spotfire functions — bookmarks, tags, lists
• Use of the text area — property controls, action controls
• Creating a user experience

Advanced Skills

• Writing expressions on the y-axis of a visualization
• Data functions, coding with TERR, and expression functions
• Action controls with IronPython snippets
• Improving the text area with HTML and CSS
• Extending the text area with JavaScript
• Optimizing project load time
• Using the predictive modeling tools

Now, it’s common to hit a crossroads when developing advanced skills. We only have so much time and learning new coding languages takes a lot of time. This is also where developing skills gets a lot harder. The more difficult and specialized the subject matter, the fewer (free) resources you’ll be able to find. For example, read this post, for an explanation of why learning IronPython for Spotfire is so hard.

Master Skills

• Learn to incorporate .NET libraries into your IronPython.
• Using the JSViz framework to create custom visualization types
• Writing Spotfire extensions in C#
• Using the SDK (Spotfire Development Kit)

Now, let’s talk about how to develop those skills.

Developing Spotfire Skills

Beginner & Intermediate

• The best way to get started is by taking a Spotfire training. “Essentials” training runs 1 – 2 days. All the vendors mentioned below will also have more advanced training available.
  • Ruths.ai offers well priced training.
  • TIBCO also has face to face and online training for different roles. I took the online training.
• TIBCO offers several free videos organized by skill level — beginner, intermediate, and advanced.
• The Ruths.ai exchange also has several free templates to help you learn.
• Go to the Data Shop Talk Archive on this website. It’s organized so you can find skills relative to how I have classified them in this post.
• Check out one of several Spotfire blogs out there. Some are more current than others.
  • https://datashoptalk.com
  • http://dataviz-ressources.com/
  • https://www.tibco.com/blog/
  • https://spotfired.blogspot.com/
  • https://blogs.perficient.com/tag/spotfire/
  • http://easyspotfire.blogspot.com/
  • https://www.datafuel.co/tag/spotfire/

Advanced

• Use Data Camp to develop Python and R skills.
• Try W3Schools to learn CSS, HTML, and JavaScript.
• The Spotfire IronPython quick reference website really helps with IronPython.
• Go to the Data Shop Talk Archive on this website.
  It’s organized so you can find skills relative to how I have classified them in this post.
• TIBCO also offers several free videos organized by beginner, intermediate, and advanced.
• Learn with my IronPython links shown in the sidebar to the right.

Master

• Try creating new visualizations with the JSVis framework, such as a donut chart.
  
• If you want to build custom extensions, you’ll need to learn C#. All of my C# references are in located here.
• You will also need to learn to use the SDK (Spotfire development kit). A good way to get started with this is to customize a company logo on the start page. 

Before I close out, I want to talk a little bit about how to choose a path. You won’t be able to do everything, so here’s something to help prioritize.

Prioritizing Development

• Learn Python/IronPython if you want to learn how to work with the Spotfire API and create a more interactive experience for the user or to do things you can’t do out of the box.
• Work with R if you need to do a lot of data wrangling and/or want to call external APIs.
• Take on HTML/CSS if you want to focus on improving the text area and making Spotfire more visually appealing.
• Learn JavaScript if you would like to create custom visualizations not offered out of the box or to make the application more interactive.
• If you are still struggling, check out this explanation of how to use different languages in the text area.

I’d love to hear other developers’ thoughts on when to learn a language for Spotfire. Please comment!

Conclusion

Now, I know some of you may have read this and thought — There is no way I will ever be able to learn all of this. It’s just too much. There isn’t enough time. The truth is that time will pass no matter what. You won’t regret spending some of it developing new Spotfire skills.

Next week, I’ll return to the data limiting series I started and start drafting part 2 of this post.