5 Big Map App Issues Apple Must Solve

The Apple iPhone 5 has been quite a story for a while with many “firsts”: the fastest hardware, the most first-week sales, and so on. The device is also now famous for what it doesn’t have: Google Maps.

5 Big Map App Issues Apple Must Solve

5 Big Map App Issues Apple Must Solve

The tech community has been aflutter over the last few days with discussions about Apple’s challenges in competing with Google and the need to improve the accuracy of Apple’s point-of-interest (POI) data. For example:

Cartographic expert Mike Dobson weighed in on the challenges that Apple faces, listing five key problems that Apple will need to solve and explaining why he doesn’t expect the company to make a quick fix of it.

Many readers do not understand why the tasks listed by Mr. Dobson are so difficult. I’ve described below how Apple might be generating its POI data for maps and provide actual examples of the challenges I’ve seen in undertaking a similar process for others.

1. Aggregating location data is hard.
For complete coverage, Apple would need to bring in two or three horizontal (core) databases for each country. For example, in the United States, a few commercial providers like Localeze, InfoGroup, Axciom, and Compass provide core data on approximately 15 million businesses. Each provider has various strengths, which is why large publishers like Apple would prefer data from more than one provider. Apple would probably want to license third-party sources in addition to collecting and curating its own data through its devices.

2. Aggregating vertical business data is even harder.
In addition to core data, Apple would need specialized databases on specific verticals. For example, Yelp for restaurants and retail locations and Hotels.com, TripAdvisor, HotelsCombined for hotels and lodging. Apple would also need retailer-specific data from Wal-mart, Starbucks, and other large national chains. When you build an app like Apple Maps, you need dozens of separate sources. It’s not uncommon for a large publisher or search engine to use more than 100 different data sources.

3. Standardizing data is a bigger process than expected.
Once Apple has received the data, it would need to process it into a common format for their team to analyze and begin integrating. However, the providers mentioned above don’t distribute their data in a standard way. Files are often provided in complex formats, such as actual database files that make the information hard to manipulate quickly.

Each source will likely have a different format for even fields as simple as the business’ telephone number. For example, some will have phone numbers in three parts that need to be assembled (area code: 212, exchange: 555, number: 1212), unformatted (2125551212), or formatted (212-555-1212 or (212) 555-1212). The case is similar with addresses, which will also be provided in different formats.

Apple would also need to standardize categories. Some providers use NAICS categories, some use SIC codes, and yet others have their own proprietary categories that Apple would need to map into a common category. In many cases there will be hundreds of different fields to standardize.

In addition to standardizing the fields, Apple would need to do more to help users find the right place. Apple would need to generate additional synonyms for each place and figure out how they relate to each other. While Apple needs to do it all, some companies, to become experts, just specialize in only one of these tasks.

Setting up the infrastructure to process these data sources is also a challenge, especially if Apple wants to do real-time updates vs. weekly or monthly batch processing.

4. When matching places, for every rule, there’s an exception.
Once the data sources have been standardized, Apple’s objective would be to assemble a composite profile for each POI by selecting and merging the best info from each data source. However, before that can be done, Apple would need to identify which profiles, within each source, refer to the same real place.

To do this, Apple would need to match the POIs to a canonical or reference place. This is hard because there isn’t a common ID that can easily link profiles to the same real place. Every provider will also have a different name, address, latitude/longitude, etc. for the same real place. For example:

Provider #1:
Days Inn Cedar Point South Milan (Ohio)
11410 State Rt 250, Milan (Ohio), Ohio

Provider #2:
Sandusky Days Inn Cedar Point South Turnpike
11410 SR 250, Milan, Ohio

Unfortunately for Apple, matching is time-consuming to analyze data, identify exceptions and improve the system. It requires a complex algorithm that needs profile details, crowdsourced info, and machine learning and reporting tools. Many parts need to be resolved in sequence so it isn’t a task where 30 developers will finish 30 times sooner than one developer.

Issues I’ve seen include:

Addresses that are difficult for a computer to understand: “Unit 6-7 The Epicurean Food Hall, Middle Abbey Street, Dublin, Ireland”
Places within places: “Grille 700 – Baltimore Marriott Waterfront” is a restaurant and shouldn’t be matched to the hotel “Marriott” at the same address.
Brand confusion: Shell Lake Esso is an “Esso” station not a “Shell” station
If Apple gets the matching wrong, it would create duplicates, as seen in this screenshot:

 

5. Merging data sources into a composite database is an entire business in itself.

To generate the information that users see in the app, Apple would need to select and merge the best data from the various sources into a composite profile for each real place. To do this, Apple would need to figure out which source has the best data for each field. Sources collected by crawling the Internet frequently have call-tracking numbers (that expire or are redirected to other companies) instead of the business’ true phone number. Sources collected by copying Yellow Page books may have details that have changed since the books were printed. If Apple chooses the wrong details, users will be navigated to the wrong location, click on broken links, see product lists that are no longer accurate, or call numbers that are no longer in service.

Advertisements

3 thoughts on “5 Big Map App Issues Apple Must Solve

  1. Pingback: Let’s Compare Open Source Or Charge Reporting Tools. | Wisdom Health Prosperity

  2. Great ¡V I should certainly pronounce, impressed with your web site. I had no trouble navigating through all tabs and related info ended up being truly easy to do to access. I recently found what I hoped for before you know it at all. Reasonably unusual. Is likely to appreciate it for those who add forums or something, site theme . a tones way for your customer to communicate. Excellent task..

  3. Excellent publish, very informative. I ponder why the other specialists of this sector don’t notice this. You must proceed your writing. I am confident, you have a great readers’ base already!|What’s Happening i’m new to this, I stumbled upon this I have found It positively useful and it has helped me out loads. I hope to give a contribution & aid different users like its helped me. Good job.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s