Crowd Sourced Tool Reviews: Can the Collaborative Economy Disrupt Technology Analyst Market? An In-Depth Guide

| September 10, 2013

Entering the Collaborative Economy Era

Altimeter Collaborative Economy

Are we entering the era of the Collaborative Economy?  If you follow the writing and research of Altimeter Partner Jeremiah Owyang you probably think so.  Jeremiah has been promoting the idea that social, mobile and payment solutions are empowering consumers to share products and services.  And companies and industries are being disrupted as consumers buy from each other rather than through traditional channels and organizations.  Altimeter’s formal definition of the Collaborative Economy is:

The Collaborative Economy is an economic model where ownership and access is between corporations, startups, and people.  This results in market efficiencies that bear new products, services, and business growth.

The classic sharing economy example is the car sharing industry.  Companies like Zip Car, Car2Go, and others provide a shared model replacing traditional individual ownership.  The car sharing model continues to evolve with other models including Lyft (peer-to-peer taxi services), Relay Rides (peer-to-peer car rental), and Carpooling.com (ridesharing).  You can marvel at the innovative uses of technology required to make these emerging sharing services possible.  However, you might then miss the key insight Jeremiah and Altimeter surface in their research: the impact on the bottom line and business models of traditional competitors.  For example, a University of California study estimates each car sharing vehicle replaces 9-13 vehicles.  The continuing rise in urban population density will likely fuel the car sharing trend and expand its impact.  Thus car sharing will have a big impact on car manufacturers.

Similar industry, technology and societal dynamics are allowing collaborative consumption models to mushroom across various industries.  Jeremiah and Altimeter provide a thorough accounting in their various research and reporting on the subject from looking at various markets being disrupted to profiles of 200 collaborative startups.  Be sure to read it all to get their full analysis.

G2 Source – The Yelp of Business Technology Reviews

An emerging Collaborative Economy model that can help digital marketers comes from G2 Crowd.  G2 Crowd was founded by technologists who became frustrated with the amount of time, effort, and money required for managing technology analyst relations.  Additionally, they wanted a better way for buyers to get the information to ease their decision making.  We all know that consumers trust recommendations from friends, family and their extended social network.  G2 Crowd bet that the same dynamics could be brought to the word of enterprise software reviews – a Yelp for business reviews says Gery Menegaz at ZDNet.

This Collaborative Economy model is aimed at disrupting the traditional, expensive technology analyst model of firms like Gartner Group, IDC, and Forrester. In the traditional model, highly paid experts opine on products, but given resource limitations (people and time) traditional sector reviews are not always timely or exhaustive.  These limitations do not even touch on the perception of bias that can creep into a model where analyst firms review client products or focus solely on the larger vendors.

G2 Crowd is focused on transparency and trust, requiring reviewers to register using their LinkedIn profiles, and preventing people from reviewing their own company’s products.  Additionally, G2 Crowd doesn’t sell leads or accept money for product placement from vendors, thus increasing perceptions on data integrity and trust.  G2 Crowd members review products they actual use or have implemented, providing a real world perspective.  G2 Crowd also uses gamification components, rewards and contests, to provide incentives for users to provide their experiences.  This has allowed them to quickly grow their membership and reviews.

G2 Crowd Reviews

G2 Crowd is now able fill out several product categories with many reviews, such as Marketing Automation, CRM, Sales, and ERP.  Additionally, there is no limit on the size of the organization reviewed.  The only limit is finding implementers willing to write a review.

Of interest to digital marketers, G2 Crowd reviews for Email Marketing, Marketing Automation, Social Marketing, Social Media Monitoring, and Web Analytics are reaching critical mass.  As Chief Marketing Officers (CMOs) and marketing departments increasingly need to become more technically savvy while buying tools directly form vendors, reviews like these will become invaluable.

The reviews themselves can be very detailed.  The G2 Crowd template allows reviewers to focus on the product and its features.  The template includes written sections on:

  • What do you like best?
  • What do you dislike?
  • Recommendations to others considering XYZ Co.

G2 Review Form

Additionally, G2 Crowd has standard ratings questions on requirements, ease of use, ease of administration, quality of support, and ease of doing business.  G2 crowd also asks implementation questions, such as was on-premise or hosted (cloud) implementation used or which edition of the tool was implemented.  Polling on pricing and terms, individual features, and integration is also included.

The results are very thorough reviews that can be leveraged by companies large and small.  The landing page for a particular tool will look familiar to anyone used to Yelp and Amazon.  A view of overall scores (on a 1 to 5 Star scale) gives an initial impression of the tools usefulness.  Individual reviews follow, so buyers can dig deeper into the pluses and minuses of the tool.  Reviews are filterable by Most Recent and Most Useful.   G2 Crowd also links to reviews of alternative tools and vendors on the right, giving buyers other choices to explore.

Review sites are only as good as the user reviews provided.  One of the frustrations with online reviews can be questions left unanswered or the need for clarification.  G2 Crowd provides the ability to contact the user.  This can be a great way to probe deeper on a reviewer’s opinions, issues or best practices.

G2_Crowd_Salesforce_Review_Example

Additionally, the polling questionnaires provide raw data that G2 Crowd can leverage.  For example, it could sell the data to vendors for impartial benchmarking.  Also G2 Crowd can apply analytics techniques to report out findings.  As Matt Gorniak, one of the founders of G2 Crowd said, “We are not analysts” in the traditional Gartner or Forrester sense in that G2 Crowd does not add any subjective input into their data analysis.  Thus, they aim to impartially analyze reviewer data without tarnishing the impartiality and trust they are building.

Look Out Magic Quadrant Here Comes The Grid

One of G2 Crowd’s first forays into data analysis looks to take on Gartner’s Magic Quadrants.  Gartner graphs vendors into four types: Leaders, Visionaries, Niche Players and Challengers.  The Magic Quadrant provides an initial and quick view to buyers looking at a new tool.  To build the chart, Gartner analysts must interview vendors, reference customers, industry contacts, and clients, as well as conduct surveys and review third party public information.  These magic quadrants obviously take time to research and create and are only updated periodically.

G2 Crowd’s alternative, or “Grids”, provides a real-time view of vendors based on the crowd sourced reviews not analyst opinion.  G2 Crowd’s analysis groups vendors into four categories: Leaders, High Performers, Contenders, and Laggards.

G2 Crowd Grid Scoring Methodology

So how does G2 Crowd come to its rankings?  According to the G2 Source website, they use the following methodology for their Grids:

G2 Crowd rates products and vendors based on ratings and reviews gathered from our user community, as well as data aggregated from online sources and social networks. We apply a unique, patent-pending algorithm to this data to calculate the product strength and vendor scale scores in real time.

The satisfaction rating is affected by the following (in order of importance):

  • Overall Customer Satisfaction and NPS Score based on ratings by G2 Crowd users
  • Customer Satisfaction with 2nd Level Product Attributes based on user reviews
  • Popularity and statistical significance based on number of reviews/ratings received on G2 Crowd

The scale score is affected by the following (in order of importance):

  • Number of employees (based on social networks and other public sources)
  • Market share based on share of voice including number of ratings and reviews received
  • Momentum based on web traffic and Google search trends
  • Social impact based on Klout score, Twitter, and LinkedIn followers
  • Age of Company (number of years in operation)
  • Employee Satisfaction and Engagement (based on social network ratings)
  • Revenue and year over year revenue growth rate (if available)

The result is a 2×2 matrix that provides a quick and easy first impression of the vendors in the desired segment.  The graph is also not static, like the Gartner version.  G2 Source updates Grids in real-time as new data becomes available.  And they provide an embeddable version that is live with the most recent data and links back to G2 Crowd allowing easy access to further information.

 

Next Step Complete Buyers Guide Reports

Now G2 crowd is taking their analysis a step further by offering segment reports for purchase.  G2 Crowd recently released its Marketing Automation Grid report.  The full report is available for a reasonable $99 price.  TMMPDX.COM was given a partial draft version of the Marketing Automation report to review for this post.

Customer Segments Served and Deployment

The report provides several demographic charts comparing marketing automation vendors on customer segment served, and deployment method.

 marketing_automation_customer_segments

Product Satisfaction Comparison

The meat of the report provides a comparison of the vendors on customer satisfaction across eight functional categories.  Do you need top of the line email and social functionality in your marketing automation tool?  Or are you more interested in campaign management and analytics?  Or do you want the highest ratings across all functions?  The bubble comparison chart provides an easy way to visually compare the vendors, while G2 Crowd also provides a table with the numerical scores if you want to dive deeper into the data.  G2 Crowd limited the marketing automation chart to those vendors with the highest score and with over 20 reviews.

marketing_automation_bubble_chart

While G2 Crowd provides the data for the purchaser to review, they do add commentary.  Again, the commentary is based solely on the analysis of the crowd sourced reviews and data.  They do not add any subjective analysis.  If you need to do even deeper due diligence for your purchase decision, detailed ratings at the feature level are available in the Grid Spreadsheets offered as premium research on the G2 Crowd site. These spreadsheets include all data on all vendors in the Grid.

G2 Crowd then provides detailed customer and adoption ratings for each of the tools.  Once you have narrowed your choice with the comparison charts, you can get further detail on your candidates. You can find information like: How likely are reviewers to recommend the tool?  How easy to use is the tool? How easy is the vendor to do business with?  G2 Crowd provides results across six satisfaction categories, as well as an overall ranking.  Additional data on satisfaction, Net Promoter Scores (NPS), and ROI is available for purchase in G2 Crowd’s Premium Research (in the Grid Spreadsheet).

Marketing Automation Vendor Profiles

This section provides profiles for the top 12 marketing automation vendors with Grid badges earned by the vendors, such as ‘Leader” or “High Performer” badges for the most highly rated vendors.  The draft provided to TMMPDX.COM only included the HubSpot profile.  The start of the profile includes an overview of the G2 Crowd user ratings, as well as comments from the most helpful (as rated by other G2 Crowd users) reviews.

G2 Crowd Quote1 G2 Crowd Quote2.jpg

Feature Ratings

Next G2 Crowd provides feature ratings.  These ratings compare HubSpot’s scores to marketing automation averages across the same eight functional categories in the bubble chart.  However, G2 Crowd then dives deeper into each function with sub-category scores.  These provide a high level of detail to digest when making your purchase decision.  For example, if campaign Management is a key functionality for you, you can see that HubSpot scores above sector averages in all campaign management subcategories, with the highest scores for event/ webinar marketing and program management.  Does this meet your use case and needs?

hubspot_campaign_management_scores

Customer Metrics

Finally, G2 Crowd provides several customer metrics in its profiles.  For example, you can see the implementation method (in-house, third party or vendor) or deployment methods (cloud or on-premise) percentages.  Other data includes a breakdown of amount of time needed to go live based on the reviews.  This shows that a majority of reviewers went live in less than a month with HubSpot.  This actual user data can give you more impartial data than you might get from a vendor sales pitch.  Discount from list price information, contract terms, and ROI details also provide powerful tools making decisions or building business cases.

Inexpensive, Impartial Third Party Data for Your Business Case or Decision Making

Overall, the G2 Crowd report looks like an inexpensive way to get smart when making a purchase decisions.  The reports provide great data for use in building a business case for budget holders, as well as easing your selection.  The $99 price for the report makes it accessible to all business sizes, especially when compared to expensive industry analyst reports.  And for larger deployments that may need to build out a more formal business case, G2 Crowd provides further details in its Grid Spreadsheets.

Best of all, all of the information is coming from peers that have actually implemented and used the tools you are interested in.  If this sounds interesting, be sure to look through the reviews on the G2 Crowd site to see what type of information reviewers are providing.  I would love to hear your thoughts and impressions.

Or if you have purchased, implemented or used a tool, help your fellow digital marketers out and provide a review!

Tags: , , , , , , , , , ,

Category: Digital Marketing, Modern Marketing Buzz, Recruit 101, Social Media, Social Media Tools, Special Agent Intermediate, tmmBosley, tmmJill, tmmKelly, TMMPDX, tmmSabrina, Veteran Advanced

Comments are closed.

https://www.tmmpdx.com/wp-content/uploads/2013/08/crowd-150x150.jpg James C. Smith Digital MarketingModern Marketing BuzzRecruit 101Social MediaSocial Media ToolsSpecial Agent IntermediatetmmBosleytmmJilltmmKellyTMMPDXtmmSabrinaVeteran Advanced ,,,,,,,,,,

Entering the Collaborative Economy Era

Altimeter Collaborative Economy

Are we entering the era of the Collaborative Economy?  If you follow the writing and research of Altimeter Partner Jeremiah Owyang you probably think so.  Jeremiah has been promoting the idea that social, mobile and payment solutions are empowering consumers to share products and services.  And companies and industries are being disrupted as consumers buy from each other rather than through traditional channels and organizations.  Altimeter’s formal definition of the Collaborative Economy is:

The Collaborative Economy is an economic model where ownership and access is between corporations, startups, and people.  This results in market efficiencies that bear new products, services, and business growth.

The classic sharing economy example is the car sharing industry.  Companies like Zip Car, Car2Go, and others provide a shared model replacing traditional individual ownership.  The car sharing model continues to evolve with other models including Lyft (peer-to-peer taxi services), Relay Rides (peer-to-peer car rental), and Carpooling.com (ridesharing).  You can marvel at the innovative uses of technology required to make these emerging sharing services possible.  However, you might then miss the key insight Jeremiah and Altimeter surface in their research: the impact on the bottom line and business models of traditional competitors.  For example, a University of California study estimates each car sharing vehicle replaces 9-13 vehicles.  The continuing rise in urban population density will likely fuel the car sharing trend and expand its impact.  Thus car sharing will have a big impact on car manufacturers.

Similar industry, technology and societal dynamics are allowing collaborative consumption models to mushroom across various industries.  Jeremiah and Altimeter provide a thorough accounting in their various research and reporting on the subject from looking at various markets being disrupted to profiles of 200 collaborative startups.  Be sure to read it all to get their full analysis.

G2 Source – The Yelp of Business Technology Reviews

An emerging Collaborative Economy model that can help digital marketers comes from G2 Crowd.  G2 Crowd was founded by technologists who became frustrated with the amount of time, effort, and money required for managing technology analyst relations.  Additionally, they wanted a better way for buyers to get the information to ease their decision making.  We all know that consumers trust recommendations from friends, family and their extended social network.  G2 Crowd bet that the same dynamics could be brought to the word of enterprise software reviews – a Yelp for business reviews says Gery Menegaz at ZDNet.

This Collaborative Economy model is aimed at disrupting the traditional, expensive technology analyst model of firms like Gartner Group, IDC, and Forrester. In the traditional model, highly paid experts opine on products, but given resource limitations (people and time) traditional sector reviews are not always timely or exhaustive.  These limitations do not even touch on the perception of bias that can creep into a model where analyst firms review client products or focus solely on the larger vendors.

G2 Crowd is focused on transparency and trust, requiring reviewers to register using their LinkedIn profiles, and preventing people from reviewing their own company’s products.  Additionally, G2 Crowd doesn’t sell leads or accept money for product placement from vendors, thus increasing perceptions on data integrity and trust.  G2 Crowd members review products they actual use or have implemented, providing a real world perspective.  G2 Crowd also uses gamification components, rewards and contests, to provide incentives for users to provide their experiences.  This has allowed them to quickly grow their membership and reviews.

G2 Crowd Reviews

G2 Crowd is now able fill out several product categories with many reviews, such as Marketing Automation, CRM, Sales, and ERP.  Additionally, there is no limit on the size of the organization reviewed.  The only limit is finding implementers willing to write a review.

Of interest to digital marketers, G2 Crowd reviews for Email Marketing, Marketing Automation, Social Marketing, Social Media Monitoring, and Web Analytics are reaching critical mass.  As Chief Marketing Officers (CMOs) and marketing departments increasingly need to become more technically savvy while buying tools directly form vendors, reviews like these will become invaluable.

The reviews themselves can be very detailed.  The G2 Crowd template allows reviewers to focus on the product and its features.  The template includes written sections on:

  • What do you like best?
  • What do you dislike?
  • Recommendations to others considering XYZ Co.

G2 Review Form

Additionally, G2 Crowd has standard ratings questions on requirements, ease of use, ease of administration, quality of support, and ease of doing business.  G2 crowd also asks implementation questions, such as was on-premise or hosted (cloud) implementation used or which edition of the tool was implemented.  Polling on pricing and terms, individual features, and integration is also included.

The results are very thorough reviews that can be leveraged by companies large and small.  The landing page for a particular tool will look familiar to anyone used to Yelp and Amazon.  A view of overall scores (on a 1 to 5 Star scale) gives an initial impression of the tools usefulness.  Individual reviews follow, so buyers can dig deeper into the pluses and minuses of the tool.  Reviews are filterable by Most Recent and Most Useful.   G2 Crowd also links to reviews of alternative tools and vendors on the right, giving buyers other choices to explore.

Review sites are only as good as the user reviews provided.  One of the frustrations with online reviews can be questions left unanswered or the need for clarification.  G2 Crowd provides the ability to contact the user.  This can be a great way to probe deeper on a reviewer’s opinions, issues or best practices.

G2_Crowd_Salesforce_Review_Example

Additionally, the polling questionnaires provide raw data that G2 Crowd can leverage.  For example, it could sell the data to vendors for impartial benchmarking.  Also G2 Crowd can apply analytics techniques to report out findings.  As Matt Gorniak, one of the founders of G2 Crowd said, “We are not analysts” in the traditional Gartner or Forrester sense in that G2 Crowd does not add any subjective input into their data analysis.  Thus, they aim to impartially analyze reviewer data without tarnishing the impartiality and trust they are building.

Look Out Magic Quadrant Here Comes The Grid

One of G2 Crowd’s first forays into data analysis looks to take on Gartner’s Magic Quadrants.  Gartner graphs vendors into four types: Leaders, Visionaries, Niche Players and Challengers.  The Magic Quadrant provides an initial and quick view to buyers looking at a new tool.  To build the chart, Gartner analysts must interview vendors, reference customers, industry contacts, and clients, as well as conduct surveys and review third party public information.  These magic quadrants obviously take time to research and create and are only updated periodically.

G2 Crowd’s alternative, or “Grids”, provides a real-time view of vendors based on the crowd sourced reviews not analyst opinion.  G2 Crowd’s analysis groups vendors into four categories: Leaders, High Performers, Contenders, and Laggards.

G2 Crowd Grid Scoring Methodology

So how does G2 Crowd come to its rankings?  According to the G2 Source website, they use the following methodology for their Grids:

G2 Crowd rates products and vendors based on ratings and reviews gathered from our user community, as well as data aggregated from online sources and social networks. We apply a unique, patent-pending algorithm to this data to calculate the product strength and vendor scale scores in real time.

The satisfaction rating is affected by the following (in order of importance):

  • Overall Customer Satisfaction and NPS Score based on ratings by G2 Crowd users
  • Customer Satisfaction with 2nd Level Product Attributes based on user reviews
  • Popularity and statistical significance based on number of reviews/ratings received on G2 Crowd

The scale score is affected by the following (in order of importance):

  • Number of employees (based on social networks and other public sources)
  • Market share based on share of voice including number of ratings and reviews received
  • Momentum based on web traffic and Google search trends
  • Social impact based on Klout score, Twitter, and LinkedIn followers
  • Age of Company (number of years in operation)
  • Employee Satisfaction and Engagement (based on social network ratings)
  • Revenue and year over year revenue growth rate (if available)

The result is a 2×2 matrix that provides a quick and easy first impression of the vendors in the desired segment.  The graph is also not static, like the Gartner version.  G2 Source updates Grids in real-time as new data becomes available.  And they provide an embeddable version that is live with the most recent data and links back to G2 Crowd allowing easy access to further information.

 

Next Step Complete Buyers Guide Reports

Now G2 crowd is taking their analysis a step further by offering segment reports for purchase.  G2 Crowd recently released its Marketing Automation Grid report.  The full report is available for a reasonable $99 price.  TMMPDX.COM was given a partial draft version of the Marketing Automation report to review for this post.

Customer Segments Served and Deployment

The report provides several demographic charts comparing marketing automation vendors on customer segment served, and deployment method.

 marketing_automation_customer_segments

Product Satisfaction Comparison

The meat of the report provides a comparison of the vendors on customer satisfaction across eight functional categories.  Do you need top of the line email and social functionality in your marketing automation tool?  Or are you more interested in campaign management and analytics?  Or do you want the highest ratings across all functions?  The bubble comparison chart provides an easy way to visually compare the vendors, while G2 Crowd also provides a table with the numerical scores if you want to dive deeper into the data.  G2 Crowd limited the marketing automation chart to those vendors with the highest score and with over 20 reviews.

marketing_automation_bubble_chart

While G2 Crowd provides the data for the purchaser to review, they do add commentary.  Again, the commentary is based solely on the analysis of the crowd sourced reviews and data.  They do not add any subjective analysis.  If you need to do even deeper due diligence for your purchase decision, detailed ratings at the feature level are available in the Grid Spreadsheets offered as premium research on the G2 Crowd site. These spreadsheets include all data on all vendors in the Grid.

G2 Crowd then provides detailed customer and adoption ratings for each of the tools.  Once you have narrowed your choice with the comparison charts, you can get further detail on your candidates. You can find information like: How likely are reviewers to recommend the tool?  How easy to use is the tool? How easy is the vendor to do business with?  G2 Crowd provides results across six satisfaction categories, as well as an overall ranking.  Additional data on satisfaction, Net Promoter Scores (NPS), and ROI is available for purchase in G2 Crowd’s Premium Research (in the Grid Spreadsheet).

Marketing Automation Vendor Profiles

This section provides profiles for the top 12 marketing automation vendors with Grid badges earned by the vendors, such as ‘Leader” or “High Performer” badges for the most highly rated vendors.  The draft provided to TMMPDX.COM only included the HubSpot profile.  The start of the profile includes an overview of the G2 Crowd user ratings, as well as comments from the most helpful (as rated by other G2 Crowd users) reviews.

G2 Crowd Quote1 G2 Crowd Quote2.jpg

Feature Ratings

Next G2 Crowd provides feature ratings.  These ratings compare HubSpot’s scores to marketing automation averages across the same eight functional categories in the bubble chart.  However, G2 Crowd then dives deeper into each function with sub-category scores.  These provide a high level of detail to digest when making your purchase decision.  For example, if campaign Management is a key functionality for you, you can see that HubSpot scores above sector averages in all campaign management subcategories, with the highest scores for event/ webinar marketing and program management.  Does this meet your use case and needs?

hubspot_campaign_management_scores

Customer Metrics

Finally, G2 Crowd provides several customer metrics in its profiles.  For example, you can see the implementation method (in-house, third party or vendor) or deployment methods (cloud or on-premise) percentages.  Other data includes a breakdown of amount of time needed to go live based on the reviews.  This shows that a majority of reviewers went live in less than a month with HubSpot.  This actual user data can give you more impartial data than you might get from a vendor sales pitch.  Discount from list price information, contract terms, and ROI details also provide powerful tools making decisions or building business cases.

Inexpensive, Impartial Third Party Data for Your Business Case or Decision Making

Overall, the G2 Crowd report looks like an inexpensive way to get smart when making a purchase decisions.  The reports provide great data for use in building a business case for budget holders, as well as easing your selection.  The $99 price for the report makes it accessible to all business sizes, especially when compared to expensive industry analyst reports.  And for larger deployments that may need to build out a more formal business case, G2 Crowd provides further details in its Grid Spreadsheets.

Best of all, all of the information is coming from peers that have actually implemented and used the tools you are interested in.  If this sounds interesting, be sure to look through the reviews on the G2 Crowd site to see what type of information reviewers are providing.  I would love to hear your thoughts and impressions.

Or if you have purchased, implemented or used a tool, help your fellow digital marketers out and provide a review!