Communication is the Key

While I have kept my hands dirty over the past six months I also know that so much more of my success is based on my ability to effectively lead and manage others.  Communication is an important component of that success.  I have noticed though that since becoming an executive the interaction I have with my team members has changed.  The frequency and level of detail when communicating with those in my organization is not what it has been in the past.  This extends to both the interaction I have with my leadership team as well as others in my organization.

Let's start with the latter where today I just do not have the same type of day-to-day conversation with everyone that is part of my team.  I realize now that as an executive I am no longer considered "one of them" or part of the rank and file and in the end probably should not be - well at least not completely.  Some level of distance is healthy - it provides my managers the ability and confidence to effectively lead.  They should be the ones developing a close relationship with their employees and not me.

As an example, although I always maintain an open door policy with anyone in my organization I am cautious when approached directly by an individual that is bypassing their manager.  I listen to what the employee has to say but in almost all cases defer to their manager trusting in their ability to resolve the issue.  To be completely honest one of the driving reasons I do communicate more with those that do not directly report to me these days is time.  I realize what a valuable commodity time is for me personally and I now use it judiciously.

As far what level of communication I have with my managers I have put quite a of thought into this as of late.  Looking back at the successful managers from my past what made them great was their ability to cultivate a strong relationship with those that worked for them.  Knowing what makes their employees tick - what motivates them and makes them happy.  The best managers always seemed to make me feel comfortable going to them with issues - those inside and and some cases outside of the office.  This only helped build a stronger bond and made me more productive.

In a recent discussion w/ our CEO he brought up another great point I hadn't considered and caused me to completely rethink how I interact with those I manage.  Responsibility he believes must also be on the employee to initiate communication.  I can't always be the one that makes sure this happens.  This fosters a healthy amount of independence with certain tasks.  Our discussion led to my realization that if I'm fulfilling my new role correctly, I ultimately become more of an arbitrator of decisions.  My team should already have considered alternatives and made a decision before coming to me.  I can then simply weigh in when competing viewpoints are raised.

Oracle Misdirection: Lies, Filthy Dirty Lies!

So I learned today that Oracle with it's EXPLAIN PLAN and AUTOTRACE features sometimes will "lie" about how a particular query is executing.

For a problematic query against a view in one of our databases, I pulled the query into Aqua Data Studio (cross platform database development/admin tool that I use)  and ran an EXPLAIN PLAN on it. What truly puzzled me was that the query appeared to be using the correct index but never seemed to return in a reasonable amount of time.

After speaking with our Senior Oracle DBA, he informed me that the RDBMS will often decide at execution to use a different plan than what EXPLAIN PLAN or AUTOTRACE says it is going to use.  Some of this decision depends on the whether or not you use bind variables or at times the data itself that you are binding in your predicates.

To troubleshoot, I needed to look for the SQL_ID for the query in question.  To make sure I pulled the EXACT query that was causing the problem, I actually looked at a join between the v$sql and v$sql_bind_capture views focusing in on a CustomerID for a query that I knew I had just run:

SELECT /*+ PARALLEL */
       v.executions,
       c.VALUE_STRING as CustomerID,
       v.sql_fulltext,
       v.sql_id,
       v.rows_processed, v.parsing_schema_name,
       v.last_active_time, v.last_load_time, v.first_load_time,
       v.disk_reads, v.buffer_gets, v.cpu_time, ROUND(v.physical_read_bytes/(1024*1024)) as read_meg
FROM  v$sql v, v$sql_bind_capture c
WHERE
v.sql_fulltext LIKE '%CustomerSummary%' AND
v.sql_id = c.SQL_ID
AND c.name like '%CustomerId%'
AND c.VALUE_STRING like '%2005681%'
AND v.PARSING_SCHEMA_NAME = 'MO_COUNTIES'
and v.last_active_time >= sysdate - 1
ORDER BY physical_read_bytes DESC;

This then gave me a SQL_ID that I could plug into the following that provided a table pinpointing the EXACT plan used at execution:

select * from table(dbms_xplan.display_cursor('3qub1pyswv17b','',''));  

Digging through this plan I discovered that an index I thought should have been used was not.  In addition to asking our DBA to gather statistics, I also for the time being modified the view such that it now has a hint driving it to the correct index.

A TNS-Nameless Connection!

This week we have been rolling out our new web application and late one evening long after our server administrator had signed off, I realized that while the Oracle client had been successfully installed on a server, the tnsnames.ora file had not been configured.

To add to my frustration, I didn't have rights to edit the file and needed to test database connectivity and verify that a view I had created to use the USERENV/HOST variable via the SYS_CONTEXT function was able to successfully differentiate inbound requests and return the appropriate data based on originating server - which of course meant I needed to test from that server in particular.

I discovered that SQL*PLUS does allow for a tnsnames-less connection by directly specifying the entry when connecting.  Below is an example of what I used to connect from the command line on the server:

sqlplus user/pw@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=myhostname.domain.com)(PORT=1521)))(CONNECT_DATA=(SID=proddb)))

This allowed me to get around not having admin rights for the Oracle client installation.  Too cool.

Searching Oracle View Source w/ Wildcard Queries

Last week I actually had the opportunity to get my hands dirty and in doing so ran into an annoyance with searching through source our custom application views within Oracle.  Because it took me longer than my customary several minute all-powerful Google search to come up with an answer that satisfied me, I decided to write a blog entry to keep track of what I did (for future reference) as well as to help others that might come across the same frustration I did.

As any respectable PL/SQL developer knows, packages and stored procedure source can be searched using a VARCHAR typed column that is provided in the ALL_SOURCE and USER_SOURCE views that are owned by SYS.  Source for views however is only exposed via a LONG column in the SYS owned DBA_VIEWS view.  The LONG data type of course is unfortunately not searchable using wildcard predicates.  For example, even the simplest query like the one depicted below that is searching for TO_CHAR function references across all views in the database returns a data type inconsistency error:

QUERY: SELECT * FROM DBA_VIEWS DBA_VIEWS WHERE TEXT '%TO_CHAR%'

ERROR: ORA-00932-inconsistent datatypes expected number got long.

I found many solutions out there that describe creating a copy of DBA_VIEWS but essentially converting the LONG column using the TO_LOB function when the new table is built.  For example, as described here you could build the copy and then query as follows:

        insert into dba_vw
      select owner, view_name, text_length,
      to_lob(text),type_text_length, type_text, 
      oid_text_length,oid_text, view_type_owner, 
      view_type, superview_name 
    from dba_views;
     
    select *from dba_vwwhere text like '%TO_CHAR%';

I didn't like this solution though as I hate creating tables as that requires extra work to get to the solution that I really want and also have to remember to drop the table after I'm done with it.  After much searching I was able to find another solution referenced here that the DBMS_METADATA.GET_DDL function to convert the data in a sub-select thereby making it available to be queried using a wildcard predicate.

    select view_name
    from ( select view_name, 
                  dbms_metadata.get_ddl ('VIEW', view_name) text
           from user_views where rownum <= 100000)
    where upper (text) like '%TO_CHAR%';

While I've found a few other references to the solution above, by in large part most of the proposed solutions go with the create a new table approach which again I didn't like and thus why I'm posting it here so that I can find it again easily and so it might benefit others as well.

Why Do Some Companies Try to Lose Customers?

This past Sunday I do something I almost never do, I actually visited FordCredit.com to try to find out exactly where I am with my loan and when I will be free and clear of having to make another a car payment.  Within seconds of bringing it up I remembered EXACTLY why I never go to their site and quite truthfully why I'll never use Ford Credit again to finance a car.

A bit of history.  So in some ways I should be a bit more sophisticated with how I remember my credentials for websites.  And yes, I know there are services out there made for people like myself that are afflicted with this totally curable disease, but let's face it, my authentication problems only really exist with sites that I don't frequent that often.  Knowing this, I tend to rely pretty heavily on the password recovery features that sites like these offer.

With FordCredit.com however (I will attempt to get that site reference in there as much as possible) they have a unique twist with their password recovery option.  In order to use it you must have your account number. This is an odd thing since:

1. I don't receive paper bills that would have the account number printed on it (unbelievably their site actually makes this suggestion).
2. Their second recommended approach for recovering your account number was to call their customer support number, enter my SSN, and retrieve my account number by listening to my account summary.  Which of course if I did that why would I even need to use their website?

I would love to know who within their engineering team had the 'fabulous' idea that my account number was such a vital piece of information to securely retrieve my credentials?  It's not as if there are not other ways to go about verifying who I am.  In fact, pretty much any site today does this by allowing users to answer security questions or in some cases (like my bank) placing a voice call to my house or text a code to my cell phone that allows me to reset my password.

It just strikes me as completely disappointing that instead of having the opportunity to form some sort of symbiotic relationship with FordCredit.com where I could have shown inherent value in going through Ford again to finance my car, instead whenever I visit FordCredit.com I am always  left with a dirty taste in my mouth and can't wait for the day when my car payments are complete.  From my perspective it really appears as if they are throwing me away as a customer.

When as a Software Developer I ask "Really?"

This morning as part of our company's new Information Assurance initiative I was required to sit through an on-line training for security policies and procedures.  For me personally it was beyond boring and in truth something we are required to do in order to meet certain governmental standards.  While I certainly understand the benefit for those that are not as "technically savvy" as myself, my only thought was how quickly could I get through the presentation and to the test awaiting me at the end of the training.

The purpose of this post however was that I had to share the image above.  Aside from the fact that the Click here link didn't do anything, it just killed me that they picked a seemingly random time period to dictate how long to wait before trying to get help.  Not 30 seconds... or even 45 seconds, but if I was stuck on the page for longer than 40 seconds then THAT is the time when I should get worried.

Build versus Buy

Over the past few weeks I've been thinking quite a bit about how software development companies, and specifically those such as ours that operate under a SaaS model, deliver their offering.  While there is obviously some grey area, in my mind companies generally fall into two camps - those that while integrating certain libraries and commercially available technologies choose to build a custom solution versus those that take a purely COTS approach.

I can remember several years ago being in an airport waiting for a flight and striking up a conversation with a Microsoft evangelist and how completely surprised I was by how the conversation unfolded.  We got into a general architecture discussion and how my company opted to build a custom horizontally scalable solution to meet our application's transactional requirements also adhering to strict up-time service agreements.  His take was that the approach we had taken was a wrong one and that if we ever wanted to mature as a business we needed to completely rethink our approach and consider using a highly integrated end-to-end COTS solution.

My counter argument at the time was the cost savings we had achieved - even today we must accommodate severe price compression in our market - think of the license savings I said rather sheepishly.  He then proceeded to rip that answer to shreds saying that in fact our custom solution cost us more in both development costs and that we couldn't go out and find engineering talent that "knew" our custom solution.  With COTS he argued, engineers that know that technology grow are easy to find.  I left that day a bit troubled with what he had said and really up until the last few weeks I wasn't able to construct a strong counter argument that makes me feel comfortable with our initial decision to go the custom route.

About the time I had my airport run-in with Mr. Microsoft, our company in fact decided to make a course change of sorts with respect to pursuing a more COTS based implementation.  We were then and still are an Oracle shop and while I was not part of those making the decision, we decided to look at how we could better utilize Oracle's product offering.  Probably the most significant system change that came out of this effort was to ditch our custom developed replication solution that had allowed both of our collocation facilities to be live (i.e., access for users could seamlessly switch between the two) and instead go with Oracle's Data Guard product in an active/standby configuration (i.e., one site would be live and the other would be a DR back-up).

The reasoning behind the decision was two fold.  First, with the custom solution there was a perception that what we had developed was "too difficult to maintain".  Developers would invariably create a new table but in the process forget to create the custom triggers necessary to keep the table in sync across sites.  As one would expect, the replication would get further and further out of sync requiring us to repair the data by hand or in some extreme cases use a feature in our storage appliance to completely re-sync the two sites from scratch.  Second, as stated by Mr. Microsoft, and supported by the many Oracle sales representatives we spoke with, if any issues did crop up engineers that knew Data Guard would be so "easy" to find.

Now roll the clock forward several years and we are still feeling the effects of this decision.  We have gone through several senior Oracle DBAs and have found none of them to have experience using Data Guard with the amount of transactions our database processes every day.  We've also paid licensing for "Active" Data Guard which is supposed to allow queries (e.g., for reports) to be issued against the standby instance only to find that the standby instance crashes after only a few hours in this mode.  And most importantly the decision cost us our true "live-live" configuration where at any point in time we knew we had a truly live secondary site where we could point users should we want to do maintenance on the primary.

Having read all of this, you might be thinking to yourself that personally I prefer the build model and you would be correct.  Now that I am the one responsible for making these sorts of key strategic decisions I look back on our change in course going with a COTS based approach and think it was a wrong one.  We gave up too much and didn't really gain anything back in return.  In practice, what I've learned is that experts on COTS solutions (especially Oracle) don't grow on trees and if you do manage to find someone talented they are prohibitively expensive.  They also aren't the typical types that want to come on as a full time employee because quite frankly they don't need to with the consulting gigs they always seem to get.

Moreover COTS offerings don't always work as advertised especially when you push them past the boundaries of what the software can do right out of the box.  Our experience with Data Guard is a prime example.  I'm pretty certain the Oracle sales representative didn't provide us any clue that we might have an issue keeping our standby in an "Active" state where we could execute queries against it.  And when you do run into these problems as we found out all too painfully your staff COTS "expert" can't think outside the box and you typically end up engaging the aforementioned prohibitively expensive consultants.

My other past experience with COTS solutions is that one typically must at least perform some amount of custom development to get them to work the way you want them to work.  Who is going to do this development you might ask?  Certainly not the COTS "expert" you hired to support the solution as they are not remotely comparable talent wise to the type of engineer that you would hire for the building a custom solution.  This is at its core why I prefer the custom build route as I believe in order to be a successful small start-up company you must first build a small team of talented innovative A-players who prefer and demand building over buying and then they in turn will go out and build you a world-class custom solution.

Never Criticize Creative Thinking

The other week while visiting on-site with one of my contractor partners I spent one late evening at the office with some of the more senior leadership of the company.  We somehow began discussing ideas that some of their more junior developers had floated.  I remember that during the discussion that we all out of hand dismissed some of their ideas concluding that while they certainly had creativity, there were real business reasons why they most likely would not find legs to stand on.

That night back at the hotel it occurred to me that in our discussions we had been too fast to pass judgment.  I actually thought of similar situations where I had experienced the same where my ideas were "poo-poohed" by senior management.  While experience in the real world can provide a more realistic outlook on what is possible to implement at the same time it can breed pessimism that in fact leads to less creative thinking.  I would say even in IT there is something magical about the innocence of youth when it comes to dreaming up truly innovative (and sometimes disruptive per an earlier post) ideas that are worth exploring.

As an example, about ten years ago at my last wireless start-up I had one of these ideas that received similar dismissive treatment.  My idea was how cool would it be to stream content from my cable box over the net to my desktop at work or wherever.  I remember vividly the reaction I received from many of the senior engineers at the company.  "Your idea will never work!" they told me, "there won't ever be enough bandwidth to support your idea".  Several years later the SlingBox was born and today a company called Aereo is taking on the network giants allowing consumers to stream over-the-air broadcasts.

That evening we also had a discussion about the usefulness of patents.  Put aside the argument of whether patents by in large are defendable - our discussion focused purely on the usefulness of them with regards to forming new ideas and lines of business.  Take a company like Yahoo for example.  If one spends time reading what they are patenting many times a typical reaction even from someone who is even seasoned from a technology perspective might be "well that is the craziest, dumbest idea I've ever heard of -  why would you even want to patent that idea?".  And sure, maybe some of them are crazy and stupid but it occurred to us that just going through the patent process itself can lead to great ideas that are not so far out of left field.

Ultimately, criticism or immediate dismissal of anyone's ideas usually results in that individual shutting off any further creative thinking.  When this 'tune-out' effect occurs it means that future great ideas are being lost from that individual.  One of our responsibilities as leaders in IT is to build environments that in fact fosters and even encourages this type of "crazy and stupid" thinking.  We always have to always ourselves that for the dozens of ideas that while they may be creative but are not unique or realistic there is that one gem of an idea that is truly original. 

Building a Winner

Let me start by saying I know that this topic has been covered countless times which would make it logical to ask what I can add that is unique or differentiating.  Even given that premise, I'm going to still try my hand at adding something to the discussion.  Building a winning team sounds so obvious, I mean who wants to build a losing team right?  But to be able to put together the right team that has chemistry, talent and drive to succeed it isn't easy.  The difference between a winning and losing team is a scarily fine line.

The other evening I was watching a documentary produced by NFL Films on the success of the San Francisco 49'ers under Eddie DeBartolo's ownership.  The documentary focused mostly on interviews with former players and coaches who tried to explain why he was so successful in building a winning organization.  There seemed to me to be two components to his success - creation of a tight-knit family friendly organization and instilling an intense desire to win.

Let's start first with his obsession to instill a winning mindset throughout his organization.  I'll lead off with a DeBartolo quote that I think perfectly sums up the type of individuals you don't want to be on your team:

Show me a good loser and I'll show you a loser.

In building those successful teams, he searched for people who just had it in their DNA a desire and passion to win.  That's not to say that those teams didn't experience failure and that even in some degree losing helped build successful teams in the future.  But he didn't want anyone around the team that didn't absolutely detest losing.  He also said that when you've brought someone on board that doesn't fit into the culture admit you've made a mistake and be prepared to address the situation by cutting that person from the team.  Admitting failure and getting rid of that individual is critically important to building a winning organization.

One might envision based on how driven he was to win that his organization was not family friendly or that there might be tension or animosity amongst employees.  Nothing could be further from the truth.  It was apparent watching the documentary that DeBartelo compensated for his brutally tough passion for winning by truly caring about everyone in his organization.  He made it a point to know every employee personally - from the head coach down to the cook in the team kitchen.  He knew their names, the names of their children, ages of their children, everything about them.  He somehow found time out of his day to make these connections with everyone - he realized that a family atmosphere within his organization was a key component to building a winner.

I think two precepts of DeBartolo's success in building a winning team are important.  Surround yourself with individuals that have the same desire you do for producing something truly special.  While they don't all have to be type-A's, it is important to find those that don't want to be on a losing team.  At the same time it's important to never forget once you have this type of an organization established how special of a situation it is and focus on making it a family by personally getting to know all of your employees and the portion of their lives that occurs outside the office.

Disruptive Innovation

Last week I traveled to participate in a series of design sessions with a .NET development studio that is building the next generation of our flagship product's website.  Over lunch one day we got into a spirited discussion regarding innovation and more specifically what it means to have innovation that is in fact disruptive.

Wikipedia defines disruptive innovation as follows:

A disruptive innovation is an innovation that helps create a new market and value network, and eventually goes on to disrupt an existing market and value network (over a few years or decades), displacing an earlier technology. The term is used in business and technology literature to describe innovations that improve a product or service in ways that the market does not expect, typically first by designing for a different set of consumers in the new market and later by lowering prices in the existing market.

Out of our discussions we came to the conclusion that it was not enough to be innovative - the innovation must also be disruptive which is a fascinating concept.  Also, in some cases a company might be responsible for what might be considered market disruption with the release of a technology that itself wasn't truly innovative.  For example, was the iPod truly a disruptive innovation by Apple?  Immediately we dismissed it as innovative product knowing there were obviously other digital media players the preceded it.  And yet, Apple and Jobs somehow took an already existing technology, and successfully transformed it into a product that appealed to the masses.

Having thought about it quite a bit, I think there are three components that contribute to a disruptive innovation in the market:

1. Being able to Envision the Disruption - One must first have the vision to be able to see how technology could enable an innovation that can be transformative and unexpected (i.e., that is disruptive).
2. Turning the Vision into a Product - Ability to design and implement a solution that satisfies the vision and results in a product that has the potential to be disruptive within the marketplace.
3. Selling the Product - And finally having business acumen to effectively market and inject the product into marketplace with widespread consumer adoption.

Very rarely do companies manage to pull off all three aspects allowing them to innovate disruptively.  Let's put this in the perspective of an example.  Many years ago people dropped their home telephone lines in favor of their cell phone - this I would say was a market disruption caused by mass adoption of the cell phone.  Let's say in the next 5 years I have a vision that with the coming prevalence of open WiFi networks coupled with carrier infrastructure build outs and market competition will produce lower data plans and lead to consumers ditching their cell phone and instead rely on a service (e.g., such as Skype) carrying with them a "smart device" that they use for placing and receiving phone calls.

Ok, I now have a vision for how disruption could be achieved by consumers employing a technology such that they no longer have to pay for both a cellular and data plan.  What next?  Even today I know I could go out and get a WiFi hotspot, set-up a tablet or other smaller device, and use a client such as Skype to provide the ability to place and receive phone calls.  This approach would let me drop my cellular plan and rely solely on my Skype service and hotspot.

Yet is this approach the right answer?  Data plans certainly are not conducive to the bandwidth requirements for extended phone calls.  Moreover, I'm not sure Skype is by itself is a service that could be a complete replacement of my cell phone (e.g., always having it on and it's effect on battery, how easy is it to receive and place calls, etc.).  And let's say for me it would be possible - would it be something consumers would widely accept over their existing cell phones?  The answer of course is no, so if not Skype, then what technology or solution would work better?  Can I take my vision and come up with a product that would allow the consumers to forgo their cell phones in the same way that they dropped their land lines?

And of course the last factor is being able to sell and market the product once produced.  Some might argue that just having a disruptively innovative product alone will cure all.  Timing of course plays an important role (ask Microsoft about innovating the tablet or Palm about touch based device you can hold in your hand) but there also is an aspect of having not only a skilled marketing and sales staff that can get the product into the hands of consumers.  I would say in addition a company's leadership must be willing to take risks on a disruptive technology.  In fact, I remember being ridiculed at a start-up in early 2000 floating an idea for a service that would allow you to stream your home cable feed over the Internet - not enough bandwidth I was told - the idea would never work.

Interestingly enough many start-up websites I've read recently speak about their company being willing to take risks on innovation.  This of course is a luxury that start-ups have and larger companies one could argue do not.  That leads me to close with an interesting twist our conversation last week took when we started talking about larger companies and whether they can innovate in the same way.  Certainly they are not able to swing for the fences as easily as a start-up and yet Microsoft with Windows8 is a recent example where a larger company has staked quite a bit on technology that could certainly cause disruption in the market.

An example on the other side might be Google's Project Glass which personally I think could have the same type of disruption but hasn't as of yet hit the market in a commercially available state.  Maybe the product was shown too early in it's development cycle or perhaps Google is electing to keep it mostly as a pilot and a higher price point believing the timing is not right for a more general release.  It is easy enough to believe that there are in fact examples in larger companies where disruptive technology remain buried in a lab somewhere never to be seen by the general public - it certainly makes one wonder what innovative technology we have missed out on?

Too Long of a Respite

I wanted to post a brief explanation of why I haven't blogged as of late.  This past year has been a long one as I've led two significant efforts at our company.  First, we decided to close one of our satellite offices where we were performing all of our software development and IT functions.  This actually proved much more difficult than I expected and at some point soon I may put together a post that details that process.  In many ways it was more difficult than any of us imagined though it has positioned our company well for the future now having all of these functions directed from underneath a single roof.

At the same time that we were undertaking that effort we also began a project to completely rewrite the website for our company's flagship product.  We outsourced this development to a truly innovative and talented .NET development house and while they ultimately produced an incredibly transformative product for a myriad of reasons (not the least of which was having to deal with the closure of our satellite office) the project went much longer than anyone expected (again, perhaps the subject of future blog post as well).  In the end however, because of their help and skills we ended up with a product that once we put the finishing touches on it will create genuine excitement in our user community and will be widely accepted.

To provide leadership for both of these efforts, I also moved into a much more senior management role during the past year.  Overall I believe it was a successful year, though it was many long hours and also pulled me away from being as hands-on as I like to be.  We now are re-focusing our efforts on the redesign of our front-end by taking what we delivered and finalizing it so it can be rolled out to our customer base.  My plan is to be much more involved in this process and I plan on ensuring that I am able to post my thoughts along the journey which IMHO is what this blog is all about.