Spatial AI vendor evaluation asks whether a location product can finish a specific job with the buyer's data, permissions, geography, and proof, rather than whether a feature list is long. A checked box for maps, AI, and security can be true while the workflow still recommends a closed place or exposes another tenant's records. The useful request names the job, the owner of each system, and the evidence required before anyone scores a vendor.
The sections below turn that request into gates a procurement team and an engineering team can share. A feature inventory still belongs in the file. The inventory answers a different question from the path a real request has to survive.
Spatial AI vendor evaluation essentials
- Start from the job: User intent, authorized data, a spatial calculation, eligibility, ranking, a validated action, and a host outcome are separate stages.
- Draw the owners: Identity, private records, transactions, and business results stay with the organization that already governs them.
- Separate credentials from permission: A platform key proves the application may call the service. End-user authorization is still the host's job.
- Rank proof, not adjectives: A claim, a document, a demo, a customer test, and a bounded pilot are different evidence levels.
- Hold the hard gates: Authorization, residency, and an unsupported critical workflow do not get averaged away by a high score elsewhere.
What Should a Spatial AI Vendor Evaluation Test?
Twelve areas cover the decision a buyer actually has to make. Workflow, architecture, data, security, geography, and the AI layer describe whether the product can perform the location job. Tools, observability, integration, reliability, cost, and a bounded pilot describe whether the organization can run that job, explain a failure, pay for it, and stop. A strong mark on a low-risk feature cannot repair a hard failure on authorization, data residency, or the workflow itself. The cover board is a map of those areas, with data sources, geospatial data, APIs, and an access boundary around the path.
Treat each area as a gate with an owner and a proof method. Workflow asks which request the product must finish, including the unhappy cases. Architecture asks which systems stay customer-owned. Data asks which records are authoritative and which fields may enter a prompt or a vector store. Security asks how tenants, roles, and private fields stay separated. Geography asks which service computes distance, containment, and travel time. The AI layer asks what the language model may interpret, and which decisions stay in deterministic code.
Tools, observability, and integration come after that foundation because they depend on it. A tool that can book, message, or write a record needs a permission check before execution. A trace needs enough identifiers to reconstruct one wrong recommendation. An integration needs a named boundary for every system the proposal touches. Reliability, cost, and the pilot then ask whether the operating model survives contact with support hours, usage drivers, and a written acceptance test. Teams that already compared building this stack with buying part of it can use that earlier split as input here, then test the bought part against the job (Kaleidr, 2026).
Why Is a Feature List the Wrong Shortlist?
A feature-first questionnaire is easy to complete with a row of yes answers. AI, maps, APIs, analytics, and security can all be present on a slide while the path from a person to a validated outcome is still undefined. The evaluation that matches production walks user intent, authorized data, a spatial calculation, eligibility, ranking, a validated action, and the outcome the host needed. Evidence belongs at every one of those stages. A vendor that checks every box on the left can still fail the path on the right.
The left panel is a feature checklist that is easy to mark yes. The right panel is the job, from user intent through a validated action to an outcome. Evidence belongs at every stage of that path. The comparison is an evaluation pattern for buyers, not a Kaleidr score.
Eligibility has to remove a closed, stale, or out-of-stock place before ranking begins. A validated action is a separate step from a sentence that describes the action. The outcome belongs to the host system that records whether the person finished the job, such as a selected place, an opened route, or a completed workflow. Ask the vendor to show those stages with the buyer's own examples, including a request that should return nothing. A demo that only shows a happy path has not yet answered the shortlist question.
Who Owns the Data, the Workflow, and the Outcome?
Draw three columns before comparing products. The customer column holds identity, tenant authorization, inventory, booking, transactions, and the business outcome. The vendor column holds the spatial layer, the SDK, ranking support, map capabilities, platform credentials, and analytics where the product actually provides them. The third-party column may hold a model provider, map tiles, routing, place data, or other cloud dependencies. Every proposal should show those boundaries, including which records never leave the customer column. Private business data stays in the systems that already govern it (Kaleidr, 2026).
The left column is what the organization already owns. The center column is the spatial platform under evaluation. The right column sits outside both, from model providers to routing and place data. The diagram asks a proposal to show boundaries, not to crown a winner.
The center column is the wrong place to relocate the system of record. A platform can rank places the customer is allowed to see, and a map can display hours or inventory the host already trusts. Pricing, contracts, payments, and the definition of a finished job stay in the customer systems that record them. Name the integration that crosses each arrow, and name the fields that stay behind it. A proposal that cannot draw this picture leaves the buyer to discover the boundary during implementation.
Why Is Platform Access Not User Permission?
Authentication proves the application can call the platform. Authorization decides which end user, tenant, role, objects, fields, and private records that request may use. A publishable browser key and a server key are platform credentials for two runtimes. The host still enforces end-user permission before private data reaches retrieval or the language model. Kaleidr's key guide separates those two credential forms (Kaleidr, 2026).
The same guide describes a publishable browser key as origin-locked and unable to act as a server bearer, and a server key as a credential for server-to-server calls that the browser rejects. Its plan table shows Pro and Enterprise keys admitting the ai, maps, and design scopes, and admitting the chat, editor, viewer, and tile products. The auth guide states that the two forms belong to the same organization and carry the same scopes, with a different runtime (Kaleidr, 2026). The Chat guide requires a key that carries the ai scope and names Pro as the minimum plan for that scope (Kaleidr, 2026). None of those facts replaces the host's own check on the person, the tenant, and the records.
The upper path proves the application may call the platform. The lower path names the person, tenant, role, objects, fields, and private data. Both paths meet before the request counts as authorized. A publishable key or a server key belongs to the upper path only.
Ask what a platform credential authorizes, and ask what it deliberately leaves alone. A key that can read a map or call a model is not a decision about which stores one employee may see. Tenant isolation, field filtering, and private-record checks belong in the host policy that runs before context is built. A prompt instruction such as "never book without permission" can guide behavior. That sentence is not an authorization layer.
Where Should a Geographic Answer Come From?
Place identity, geometry, spatial services, and routing form the geographic foundation. Eligibility then asks whether a location is valid for this user and this moment. Ranking chooses among the options that remain. An AI layer can interpret the request and explain the result after those steps. The language model is not the routing engine, and a stack diagram exists so those jobs stay apart.
Travel time, distance, containment, and nearest-place calculations need a test the buyer can rerun. A travel time drawn on a slide is an illustration, not a measured result from the buyer's cities. Ask which service computed the route, which place identifier was canonical, and which candidates eligibility removed. Repeat the test after a model or data update, and segment it by market when the business operates in more than one. The accuracy guide separates those checks from a single overall quality claim (Kaleidr, 2026).
The lower layers represent and measure places. Routing computes movement, and eligibility decides which locations remain valid. Ranking then orders only those locations, and the AI layer interprets and explains. The stack is a job split, not a product trophy.
A buyer should also ask where each layer runs. Coordinates and boundaries may come from the customer's own places. Distance and containment may come from a geospatial service with a published method. Travel time may come from a routing provider with its own freshness limits. If the proposal treats all of those as one model answer, the evaluation has not yet found the calculation it needs to test.
How Do You Tell a Claim From Proof?
Retrieved text is a common way a spatial product goes wrong, because place descriptions, uploads, partner feeds, and web sources are not instructions. OWASP's 2025 excessive-agency entry describes damaging actions that follow unexpected, ambiguous, or manipulated model output, and it names excessive functionality, permissions, and autonomy as typical causes (OWASP, 2025). The vector and embedding entry separately describes unauthorized access to embeddings, data poisoning from insiders or unverified providers, and cross-context leakage when tenants share a vector store (OWASP, 2025). OWASP labels those risks LLM06:2025 Excessive Agency and LLM08:2025 Vector and Embedding Weaknesses. Ask for the architecture, the tests, and the residual risk. A yes to "protected from prompt injection" does not answer that request.
Sort every important answer onto an evidence ladder. Level 0 is the word supported. Level 1 is documentation that describes the behavior. Level 2 is a vendor demonstration in a controlled environment. Level 3 is a customer test with the buyer's integration or data. Level 4 is a bounded pilot with measurable acceptance criteria. Higher-risk requirements demand a higher level, and a marketing sentence does not equal a reproduced test.
The ladder climbs from a claim to documentation, a vendor demo, a customer test, and pilot evidence. Higher-risk requirements should demand a higher rung. A checked word on a questionnaire sits on the bottom rung. The ladder is a scoring rule for buyers, not a Kaleidr result.
NIST's AI RMF Core puts third-party software, hardware, and data inside the issues the Govern function has to cover, and the same page notes that the AI RMF 1.0 is being updated (NIST, 2023). On July 8, 2026, NIST announced a finalized Cybersecurity Supply Chain Risk Management Due Diligence Assessment Quick-Start Guide. The announcement says assessments start with due diligence, and that acquirers need to understand supplier risk before a procurement decision is executed (NIST, 2026). The guide is not a Spatial AI standard. The due-diligence step still applies to model, map, route, and place-data suppliers that sit inside a production stack. Ask which of those suppliers can receive customer data, how a change is communicated, and what happens if a critical supplier fails.
What Should the RFP Ask the Vendor to Prove?
Write the requirement beside the test and the evidence, so scoring has a method before it has a number. Two rows show the shape. Private records must respect user authorization: give two users different permitted locations, and require that unauthorized places never appear in retrieval, model context, the map, or logs. Invalid recommendations need closed, stale, and out-of-stock cases, with correct eligibility and a defined behavior when nothing qualifies. A requirement that cannot name its proof method is not ready to score.
Each row pairs a requirement with the test and the evidence. One row covers private records and two users with different permitted locations. The other covers closed, stale, and out-of-stock places, plus a defined result when nothing qualifies. The table is a writing pattern, not a completed Kaleidr audit.
The same sheet should demand known limitations, material third parties, and a way to leave. Unsupported geographies, missing export paths, rate limits, and freshness assumptions are useful answers, because a hidden limit costs more during rollout than a stated one. Observability should reconstruct one wrong recommendation from authorization through retrieval, geography, ranking, and the host outcome (Kaleidr, 2026). Business value belongs in a separate measurement of that outcome, such as a completed selection or a finished workflow, rather than in a count of features (Kaleidr, 2026). Cost questions belong here too: name the usage drivers and the enterprise service fees before the pilot, and refuse a score that averages a residency failure into a win on a diagram.
How Does Kaleidr Map to This Evaluation?
Kaleidr's developer docs describe four product surfaces on one platform. Chat is Spatial AI in the host's map. Editor is draw and edit. Tile serves designed basemaps. Viewer publishes a map (Kaleidr, 2026). The Viewer guide says an embed uses a share id, exchanges no key, and can be used by anyone who has that share id, including on the Free tier. Passing a key into a Viewer embed is rejected (Kaleidr, 2026). Those surfaces are concrete items an enterprise team can check against the credential and product gates above. The list does not say that every adjacent system ships inside the same product.
The host remains the owner of users, tenant authorization, private business data, workflows, transactions, and business outcomes. Map renderers, routing, and place data may be third-party or host dependencies, depending on the deployment. Kaleidr Enterprise is the location-intelligence API and map SDK surface for that integration. The mapping is a boundary diagram. The host keeps the CRM, the inventory, the booking system, and the payment stack, and Kaleidr has to fit beside them.
The left column stays with the host product. The center column lists documented Kaleidr surfaces, from Chat and Editor to Tile, Viewer, and analytics. The right column holds renderers, business systems, and external routing or place data. Documented access splits a publishable browser key, a server key, and a Viewer share id.
Read the center column against the docs, not against a wish list. Chat attaches Spatial AI to a map the host already runs, and that surface is the one that requires the ai scope. Editor mounts drawing and editing in the host product. Tile serves a designed basemap. Viewer embeds a published map by share id and does not take a platform key. Analytics, where the deployment includes it, is a usage and engagement surface. Any of those can be in or out of scope for a given RFP. The diagram's job is to make that scope visible before a pilot starts.
What Should Happen Before the Deployment Scales?
Define the job and the hard gates first, then spend the pilot on the requirements that would be expensive to discover later. Use the buyer's data, the buyer's places, and a written acceptance test for authorization, eligibility, and no-result behavior. Expand geographies, actions, and user groups only after that evidence exists. A scale decision follows from the combined system, which means the vendor response, the buyer's tests, and the pilot result together. Explore Kaleidr Enterprise when the evaluation needs a spatial layer beside systems the organization already runs, and read Enterprise Spatial AI Pilot Before Scaling for how to keep that pilot bounded.
Note: Kaleidr uses AI-assisted tools for image creation, content refinement, and research throughout its creative and development workflows.
FAQs
Does a longer feature list win a spatial AI vendor evaluation?
No. AI, maps, APIs, analytics, and security can all be present while eligibility, authorization, or the host outcome still fails. Score the path from the request to a validated result, and keep a hard failure on authorization, residency, or the critical workflow from being averaged away.
Is a platform API key the same as end-user permission?
No. A publishable browser key or a server key proves the application may call the platform. The host still decides which person, tenant, role, and private records that call may use. Kaleidr documents those keys as two runtimes of the same organization scopes, and documents Viewer embeds as share-id links that do not exchange a key.
Can a language model replace routing and eligibility?
No. Place identity, geometry, spatial calculations, and travel time need a testable geographic service. Eligibility removes invalid places before ranking. A language model can interpret the request and explain the result after those steps.
What evidence should a high-risk requirement demand?
Demand a customer test or a bounded pilot, not a sentence that says supported. Documentation and a vendor demo are useful lower rungs. Authorization, tenant isolation, and residency should sit on a higher rung, with the buyer's own cases and a written acceptance check.
Does Kaleidr replace the systems around the map?
No. Current docs describe Chat, Editor, Tile, and Viewer as spatial and map surfaces, with publishable keys, server keys, and Viewer share ids for access. Identity, private business data, inventory, booking, payments, and the business outcome stay with the host. The evaluation asks whether those surfaces fit the architecture the organization already runs.
References
- Kaleidr. Build vs Buy Spatial AI. https://kaleidr.com/blog/build-vs-buy-spatial-ai
- Kaleidr. Grounded Spatial AI for Business Data. https://kaleidr.com/blog/grounded-spatial-ai-business-data
- Kaleidr Developer Docs. Get an API Key. Publishable browser keys and server keys, and the plan table for scopes and products. Accessed October 2, 2026. https://docs.kaleidr.com/get-an-api-key
- Kaleidr Developer Docs. Auth & Scopes. Two key forms, same organization and same scopes, different runtime. Accessed October 2, 2026. https://docs.kaleidr.com/platform-api/auth-and-scopes
- Kaleidr Developer Docs. Chat. The key must carry the ai scope, and Pro is the minimum plan for that scope. Accessed October 2, 2026. https://docs.kaleidr.com/chat
- Kaleidr. Spatial AI Accuracy Evaluation. https://kaleidr.com/blog/spatial-ai-accuracy-evaluation
- OWASP Gen AI Security Project. LLM06:2025 Excessive Agency. Damaging actions from unexpected, ambiguous, or manipulated model output; excessive functionality, permissions, and autonomy. Accessed October 2, 2026. https://genai.owasp.org/llmrisk/llm062025-excessive-agency/
- OWASP Gen AI Security Project. LLM08:2025 Vector and Embedding Weaknesses. Unauthorized access to embeddings, data poisoning, and cross-context leakage in a shared vector store. Accessed October 2, 2026. https://genai.owasp.org/llmrisk/llm082025-vector-and-embedding-weaknesses/
- National Institute of Standards and Technology. AI RMF Core. Govern covers third-party software, hardware, and data. The page notes that the AI RMF 1.0 is being updated. Accessed October 2, 2026. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/
- National Institute of Standards and Technology. NIST Releases Finalized C-SCRM Due Diligence Assessment Quick-Start Guide. July 8, 2026. Acquirers need supplier-risk information before procurement, and assessments start with due diligence. Accessed October 2, 2026. https://www.nist.gov/news-events/news/2026/07/nist-releases-finalized-c-scrm-due-diligence-assessment-quick-start-guide
- Kaleidr. Spatial AI Observability. https://kaleidr.com/blog/spatial-ai-observability
- Kaleidr. Spatial AI ROI Business Case. https://kaleidr.com/blog/spatial-ai-roi-business-case
- Kaleidr Developer Docs. Products. Chat is Spatial AI in the host map, Editor is draw and edit, Tile is designed basemaps, and Viewer publishes a map. Accessed October 2, 2026. https://docs.kaleidr.com/
- Kaleidr Developer Docs. Viewer. An embed uses a share id, exchanges no key, and rejects a key passed into the embed. Accessed October 2, 2026. https://docs.kaleidr.com/viewer
- Kaleidr. Location Intelligence APIs and Map SDK. Accessed October 2, 2026. https://kaleidr.com/enterprise
- Kaleidr. Enterprise Spatial AI Pilot Before Scaling. https://kaleidr.com/blog/enterprise-spatial-ai-pilot-before-scaling
@misc{kaleidr_build_vs_buy_rfp_2026,
title = {Build vs Buy Spatial AI},
author = {{Kaleidr}},
year = {2026},
url = {https://kaleidr.com/blog/build-vs-buy-spatial-ai}
}
@misc{kaleidr_grounded_rfp_2026,
title = {Grounded Spatial AI for Business Data},
author = {{Kaleidr}},
year = {2026},
url = {https://kaleidr.com/blog/grounded-spatial-ai-business-data}
}
@misc{kaleidr_docs_api_key_2026,
title = {Get an API Key},
author = {{Kaleidr}},
year = {2026},
url = {https://docs.kaleidr.com/get-an-api-key}
}
@misc{kaleidr_docs_auth_scopes_2026,
title = {Auth & Scopes},
author = {{Kaleidr}},
year = {2026},
url = {https://docs.kaleidr.com/platform-api/auth-and-scopes}
}
@misc{kaleidr_docs_chat_2026,
title = {Chat},
author = {{Kaleidr}},
year = {2026},
url = {https://docs.kaleidr.com/chat}
}
@misc{kaleidr_accuracy_rfp_2026,
title = {Spatial AI Accuracy Evaluation},
author = {{Kaleidr}},
year = {2026},
url = {https://kaleidr.com/blog/spatial-ai-accuracy-evaluation}
}
@misc{owasp_llm06_2025,
title = {LLM06:2025 Excessive Agency},
author = {{OWASP Gen AI Security Project}},
year = {2025},
url = {https://genai.owasp.org/llmrisk/llm062025-excessive-agency/}
}
@misc{owasp_llm08_2025,
title = {LLM08:2025 Vector and Embedding Weaknesses},
author = {{OWASP Gen AI Security Project}},
year = {2025},
url = {https://genai.owasp.org/llmrisk/llm082025-vector-and-embedding-weaknesses/}
}
@misc{nist_ai_rmf_core_2023,
title = {AI RMF Core},
author = {{National Institute of Standards and Technology}},
year = {2023},
url = {https://airc.nist.gov/airmf-resources/airmf/5-sec-core/}
}
@misc{nist_cscrm_quickstart_2026,
title = {NIST Releases Finalized C-SCRM Due Diligence Assessment Quick-Start Guide},
author = {{National Institute of Standards and Technology}},
year = {2026},
url = {https://www.nist.gov/news-events/news/2026/07/nist-releases-finalized-c-scrm-due-diligence-assessment-quick-start-guide}
}
@misc{kaleidr_observability_rfp_2026,
title = {Spatial AI Observability},
author = {{Kaleidr}},
year = {2026},
url = {https://kaleidr.com/blog/spatial-ai-observability}
}
@misc{kaleidr_roi_rfp_2026,
title = {Spatial AI ROI Business Case},
author = {{Kaleidr}},
year = {2026},
url = {https://kaleidr.com/blog/spatial-ai-roi-business-case}
}
@misc{kaleidr_docs_home_2026,
title = {Kaleidr Developer Docs},
author = {{Kaleidr}},
year = {2026},
url = {https://docs.kaleidr.com/}
}
@misc{kaleidr_docs_viewer_2026,
title = {Viewer},
author = {{Kaleidr}},
year = {2026},
url = {https://docs.kaleidr.com/viewer}
}
@misc{kaleidr_enterprise_rfp_2026,
title = {Location Intelligence APIs and Map SDK},
author = {{Kaleidr}},
year = {2026},
url = {https://kaleidr.com/enterprise}
}
@misc{kaleidr_pilot_rfp_2026,
title = {Enterprise Spatial AI Pilot Before Scaling},
author = {{Kaleidr}},
year = {2026},
url = {https://kaleidr.com/blog/enterprise-spatial-ai-pilot-before-scaling}
}