Financial AI in-house: who manages customer data and wrong answers?

This article provides general information based on public sources and is not legal advice. Individual data-handling and contractual decisions require examination of the actual work and rules with a qualified legal professional.

What Is Happening

Read adoption through the scope of the Bank of Japan's 2026 survey

As financial institutions use generative AI for more work, they face a practical question: what customer information may be entered? Running AI inside an organization, without sending its working data to an external service, is one option. But keeping information within a boundary does not establish that answers are correct or that only authorized people can retrieve it.

The Bank of Japan surveyed 150 of its counterparty financial institutions[1] in 2026. Its report（外部サイト、新しいタブで開きます） states that 79.3% were using generative AI[2], while 11.3% were conducting trials[2]. The reach of use and testing together does not mean every financial institution has deployed a customer-facing service in production.

The survey also distinguishes internal tasks, such as drafting and transcription, from directly presenting AI output to customers. It reports caution toward the latter. Adoption alone does not reveal which data a system uses, whose judgment it supports, or who receives its answers. Understanding the current position requires both adoption figures and the conditions of particular tasks.

Background & Context

Separate deployment location from data handling in official Japanese materials

Location Does Not Settle the Question

A local LLM is a configuration that runs a large language model on equipment within an organization or a similar local environment. The model generates text from input, but fluent writing does not guarantee factual accuracy. An on-premises configuration, operated on the organization's own equipment, can help control transmission routes. It does not by itself eliminate wrong answers or unauthorized access.

Japan's Financial Services Agency separates personal information in prompts, outsourcing, and relationships involving overseas providers in its AI discussion paper, version 1.1（外部サイト、新しいタブで開きます）. The document presents issues for future dialogue; it does not prescribe a uniform set of specific responses for every institution.

For customer information entered into a generative AI service, the paper explains that the provider normally needs to be managed as an entity entrusted with handling personal data. Service selection and contracts should address matters including whether additional training occurs without the institution's instructions. This examines the purpose of processing and who handles the data, rather than deciding solely that cloud use is prohibited or in-house use is permitted.

External service

Examine the provider relationship

Data

Scope and purpose

Provider

Contract and training

Connections

Destinations and controls

In-house operation

Examine the operator's scope

Data

Sources and permissions

Operator

Updates and failures

Answers

Evidence and corrections

A reading framework based on FSA version 1.1, pages 31 to 32, 36, and 38 to 39. External services also need answer checks; in-house systems also need provider oversight when work is outsourced. Neither choice removes the other responsibilities.

What No Training Use Establishes

The Personal Information Protection Commission's alert of June 2, 2023（外部サイト、新しいタブで開きます） asks organizations to check whether entering personal information is necessary to achieve the specified purpose of its use. It warns that handling personal data entered without the person’s prior consent for purposes other than producing the response can potentially violate the law, and includes checking that the data is not used for machine learning.

Consequently, a no-training commitment does not complete data governance. Questions remain about necessity, who processes the information, and where it is passed. The FSA paper also treats the relationship with a foreign provider separately from server location. A domestic server address alone does not settle a contract or the conditions of data handling.

One evaluation example is to consider the same question under different staff permissions. If an employee asks about customer documents outside their assigned work, examine whether answers, citations or search results reveal material they cannot otherwise access. This is a proposed check, not a report of a breach at a financial institution. Retrieving only permitted documents and producing an answer suitable for the task remain different questions.

Consider information that does not appear in the answer as well: prompts and records retained for examining use. More records should not automatically count as stronger protection. Identify who may inspect them and for what purpose, while avoiding wider circulation of unrelated customer details. Particular retention arrangements require examination against the actual work and applicable internal rules.

In-house configurations likewise require task-specific access permissions and an allocation of operational work. If an employee can obtain a document through an AI answer that they could not otherwise view, preventing external transmission would not satisfy the intended information controls. Choosing a location is one part of defining processing and access boundaries.

Reading the Structure

Examine a design-document trial, a shared-platform plan, and operational responsibility

What a Design-Document Trial Shows

NTT DATA INTELLILINK's announcement of April 14, 2026（外部サイト、新しいタブで開きます） reports a local-LLM trial in an on-premises, closed environment with NTT DATA Financial Technology. This proof of concept (PoC) examined application to financial-system development in a configuration that did not transmit the working data outside the environment.

The proposed uses were consistency checking, improving document quality, and accelerating revisions. The use tested in this announcement was consistency checking for design documents; the other uses were described as future work. The company reports a reduction in review steps, but the public page does not give a reduction percentage or detailed numerical comparisons. It is possible to acknowledge the reported result while keeping its implications within the published scope.

A design-document trial also cannot establish that AI can safely determine lending eligibility or provide financial-product guidance. The documents, consequences of errors, and recipients are different. Before extending a trial, agree what counts as a correct finding. For a consistency review, the reviewer could examine which passages conflict and whether revision is needed. More flagged passages do not by themselves show higher quality. Include time spent checking false alerts and inconsistencies the system missed. Separating draft-generation time from the time needed to complete human review helps examine whether burden moved to a later stage.

Moving to actual work requires examination of the conditions under which results can be reproduced, how errors are discovered, and who corrects them.

Published trial

Design-document consistency

Environment

On-premises and closed

Material

Development documents

Evidence

Company's report

Operational deployment

Assess each use separately

Evaluation

Compare the same task

Checks

Find and correct errors

Responsibility

Update and stop use

The left side follows the company's April 14, 2026 announcement. The right side is this article's evaluation framework, not a claim that all checks were completed in the trial or that a bank deployed it in production.

Sharing Infrastructure Still Requires Decisions

NTT DATA's May 19, 2026 announcement（外部サイト、新しいタブで開きます） describes building a shared generative AI platform for financial institutions, with provision planned to begin at the end of fiscal 2026. The plan includes controls for input, output, and usage records. A planned service start needs to remain distinct from demonstrated operation.

Sharing infrastructure may offer opportunities to share equipment and operational work, but decisions still need owners. The FSA paper describes the challenge of allocating responsibilities among developers, maintainers, and multiple contractors. Using a common platform does not imply that customer data or model intellectual property may be shared freely. Expected benefits need examination alongside contracts and information boundaries.

Costs likewise extend beyond model usage charges. The April trial describes an aim of making costs more predictable, rather than a comparative result establishing inexpensive deployment for everyone. This article proposes comparing the burden of continuing the same work, including equipment, updates, staff checking time, and recovery from failure.

For shared-platform updates, consider changes to the model separately from changes to reference material. Records of those changes could help examine why an answer to the same question differs. Connect the contact receiving a notification with staff using the output, and discuss what to check before continuing. Where participating institutions have different rules, distinguish common checks from matters each institution needs to assess.

Retrievable Does Not Mean Correct

Retrieval-augmented generation (RAG), which searches internal documents and uses retrieved material to generate an answer, can make supporting sources easier to examine. However, the FSA paper explains that false output may remain when source preparation or workflow design is inadequate. Finding a document and answering in line with current rules are separate achievements.

The BOJ survey also identifies room for improvement or continuing consideration in governance arrangements, third-party risks, and security, alongside work on information controls and practical rules. Management continues after usage grows. Operators need to examine whether staff can check the supporting material and who tracks outdated documents or model changes during operation as well as at introduction.

After discovering an incorrect answer, revising the text may leave further work. Distinguish an internal draft that went no further from an answer used in operations or delivered to a customer, then determine where a correction needs to go. Consider communication between those stopping AI use and those continuing normal work. These are questions for planning operations, not reports of an incident or a specific institution’s response procedure.

Where earlier versions remain searchable under the same rule name, consider showing which version and effective period supported an answer. Questions without sufficient supporting material could return to a staff consultation with the original documents, rather than being completed through speculation.

Related Guide

Designing outcome indicators（このサイトの記事） separates the amount of activity from the change it produces. The distinction also helps evaluate AI usage counts against improvements for staff and customers.

Reference Books

AIガバナンス入門: リスクマネジメントから社会設計まで (ハヤカワ新書) (Introduction to AI Governance: From Risk Management to Social Design; title translated)（外部サイト、新しいタブで開きます） by 羽深宏樹 is a 2023 related-reading option. Hayakawa’s public introduction（外部サイト、新しいタブで開きます） describes a book about AI benefits, risks and social design. The book itself is unread here; it does not establish the effectiveness of financial-institution operations or incident responses.

The photograph is an AI-generated server-room illustration, not a photograph of an actual financial institution's equipment.

Remaining Questions

Ask whose workload changes and who can approve changes or stop use

When an institution reports that AI reduced work, whose task became shorter? Faster drafting might coexist with more corrections by reviewers or more time explaining revisions to customers. Counts of processed documents and usage alone cannot settle the change in overall burden. Can evaluation follow checking time, corrections, and inquiries for the same task? Staff also need a way to report additional work, or the account of benefits may omit the burden transferred to them.

A shared platform raises another question: who agrees to changes in models and source documents? A useful change at one institution may conflict with another's rules or customer processes. Can participating institutions defer an update, examine its effects, and return to an earlier state where needed? A provider's change notification and an institution's understanding of the consequences for its own work are different things to verify.

The deployment decision should extend to who can stop use and provide explanations. If an incorrect answer reaches a customer, can staff halt the system, identify the affected scope, and deliver a correction? With limited staffing at regional institutions, which work can be shared and which decisions should remain with each institution? Evaluation should include the conditions for responding when customers and staff encounter problems, beyond whether the institution installed an AI system. Do examination records themselves circulate customer information more widely than necessary? After a correction is sent, can the institution check conditions that would prevent staff from continuing to use the earlier answer?

References

Publication of the AI Discussion Paper, Version1.1 (title translated) — Financial Services Agency (2026). March 3, 2026 announcement

AI Discussion Paper, Version1.1 (title translated) — Financial Services Agency (2026). Pages 6, 31–32, 36, 38–39, and 49: status, data governance, providers, incorrect output

Generative AI use and risk management at financial institutions: FY2026 survey results (title translated) — Bank of Japan, Financial System and Bank Examination Department (2026). August 6, 2026; printed pages 2–3, 5, and 9: survey scope, uses, and risk management

Alert concerning the use of generative AI services (title translated) — Personal Information Protection Commission (2023). June 2, 2023; page 2, precautions for personal information handling businesses

NTT DATA INTELLILINK tests generative AI for financial-system development in a local-LLM environment (title translated) — NTT DATA INTELLILINK (2026). April 14, 2026; scope of the trial and future use cases

NTT DATA begins building a shared AI platform for financial institutions (title translated) — NTT DATA (2026). May 19, 2026; construction and planned provision from the end of fiscal 2026

Statistics cited in this article

1. 1Bank of Japan FY2026 generative AI survey, printed page 2（FY2026） Open source
2. 2Bank of Japan FY2026 generative AI survey, printed page 3（FY2026） Open source

Related Content

October 22, 2026

Whose Act Is an AI Transaction? Reading the First Council’s Draft Issues

An analysis of agency-related acts, responsibility, permissions and audit logs in the first council’s October 22 papers. It distinguishes draft issues and members’ views from enacted rules and connects intended authority to recorded execution.

Digital & AIAIGovernance

October 16, 2026

Gennai OSS: Is Published Code Enough for Deployment?

The April 2026 partial release, the release planned around February 2027, and the license, data, and maintenance checks for adopting organizations

Digital & AIAIGovernance

March 4, 2026

Introducing 'AI Fluency for Nonprofits': A Free AI Literacy Course for NPO Staff

Introducing the free AI literacy course for nonprofit professionals by Anthropic and GivingTuesday. A practical skills program based on the 4D Framework.

ResourceAINPO

October 19, 2026

Does a Shorter Governance Code Reduce Directors’ Responsibilities?

Japan’s July 2026 principles and interpretive guidance, Companies Act duties, board information and the July 2027 report deadline.

GovernanceEconomyPublic Policy

Get new columns by email

1-2 social structure analysis columns per week. Free to subscribe.

Website

Subscribe→

Join ISVD's activities?

Sign up to receive the latest research and activity reports. Feel free to reach out about collaboration or project participation.

Join→Contact Us→

Support ISVD’s research and activities

Support ISVD’s research and publications with JPY5,000 per year. There is no automatic renewal.

Become a supporting member→About supporting membership→