Kayak—the travel search site that aggregates information on hotels, flights, car rentals, and yes, now trains—from many sources is always futzing. Should photos run here or there? Should this button be green or orange? Should the font be Ariel or Helvetica?
This makes sense because very small changes can have a measurable impact on user response rates, chief technology officer Giorgos Zacharia told Coins2Day during a recent meeting at the company’s office in Kendall Square, a technology hub in Cambridge, Mass., just outside of Boston.
“We run tens of experiments at any one time, so when a user comes to Kayak, it’s a coin flip which feature they’ll see. Each idea can run for several weeks or sometimes a few days,” said Zacharia.
None of that is unusual for a web company where fast tweaks are necessities: If you get five paid ad conversions when the button is yellow—but 50 when it’s green—the decision is obvious.
Once an idea is hatched, it is subjected to live and onscreen testing. A button color may change for X number of users coming to the site, and their interactions are gauged against users who still see the old color. Whichever gets the best result in terms of more clicks, longer time spent on the site, etc. Wins the day. This is commonly known as A/B testing, a process that serves up different versions of the same web page to users and tracks the subsequent interactions anonymously. Every click is tracked and catalogued. This is done in the background, but there are also huge screens in the company’s theater that show live interactions.
What is different about Kayak is its use of SWAT teams to bring more ideas—and thus more tests—to the table. Every staffer—not just the techies—are encouraged to contribute. Zacharia borrowed the term from law enforcement as SWAT stands for Special Weapons And Tactics. These are the teams called into resolve tense situations, save hostages, and deal with armed criminals. (You know the drill.)
The company said experiments (or XPs, as Kayak likes to call them) like this have boosted revenue by 12% this year. However, Kayak, a unit of Priceline (PCLN) since 2012, does not break out its revenue figures.
Get Data Sheet, Coins2Day’s daily technology newsletter.
For a year now, Zacharia has assembled teams from all parts of the company, taken them out of their normal projects and deadlines, and turned them loose on a tough problem. In April, Kayak inaugurated a SWAT Week competition during which the whole company was divvied into teams to brainstorm ideas. The group that generated the biggest revenue lift won $10,000. Zacharia would not detail the winning entry—this is a competitive industry after all—but he did say the idea helped get more users visiting to the site for flight information to add hotel reservations to their orders.
“We are mostly known for booking flights, but half our revenue comes from hotel bookings,” Zacharia noted.
One of the most fruitful changes Kayak made was switching the button text on the bookings page from “Book,” to “Select,” to “View.”
The theory is that when a would-be customer sees the term “Book,” it indicated a degree of commitment for which the customer might not be ready, Zacharia acknowledged. “Select” was seen as less prescriptive, and “View” has been deemed completely neutral. Boom! Click rates soared.
For more on online travel, watch:
A SWAT team also developed Kayak’s Amazon “skill,” a bit of software which lets users with Amazon’s (AMZN) Echo devices ask questions like, “Alexa, where can I go this weekend for $200?”
Another change made at the suggestion of a non-technical employee was to move the hotel photos from the right hand side of the page to the left. They tried the switch, click rates soared, and now it’s the law of Kayak land—at least until a better idea comes along.