Artificial Intelligence (AI) is moving faster than most playbooks were written to address. The organizations getting it right aren’t the ones with the biggest technology budgets; they’re the ones that stayed clear about what they were trying to accomplish, applied ethical guardrails from the start, and kept people at the center of every decision.
At its simplest, AI refers to technology that can complete tasks that usually require human thinking. This includes recognizing patterns, making predictions, generating written content, and helping people make sense of large volumes of information. In health and human services, AI can help support care coordination, training, documentation, quality improvement, data analysis, and population health planning (U.S. Department of Health and Human Services [HHS], 2025).
AI can scan thousands of records, identify individuals who may be at risk, draft summaries, or even respond to a client’s question at 2am when no one else is available. It’s not magic, and it’s not infallible. But when used thoughtfully, it’s already making a real difference in how care is delivered.
AI is not a future concept; it’s already in the room including with our clients. The people we serve are already using AI. They are asking AI questions about their medications, their diagnoses, their legal rights, and their treatment options. This isn’t a problem to be solved. It’s a reality we need to understand. Our role is to meet clients where they are at and bring what AI can’t; human judgement, clinical expertise, compassion, and relational trust.
On our end, AI is showing up in tangible ways. Risk stratification tools can identify individuals who may need to be outreached before a crisis occurs. Care coordination dashboards provide real-time visibility into complex situations. Documentation tools reduce administrative burden, allowing staff to spend more time with the people they serve.
AI brings exciting possibilities, but it also brings real responsibility.
Why Ethical AI Matters
Ethical AI begins with one important question: Who could be helped, and who could be harmed?
AI can help staff work more efficiently, but it should support human decision-making, not replace it. A person’s life cannot be fully understood through a score, a form, or a data field. AI can identify patterns, but it does not understand trauma, fear, family dynamics, culture, lived experience, or understand the hesitation in someone’s voice when they ask for help.
The heart of this work remains human: judgment, compassion, and relationship-building.
Where Organizations Need to Be Careful
AI becomes risky when it is used without proper oversight or when it affects someone’s access to care, services, or support. Areas that require particular caution include eligibility decisions, risk scoring, referrals, care planning, referrals, documentation, prioritization, and service access.
AI-generated information can be wrong, even when it sounds confident. It can leave out important context or misrepresent a situation. In human-centered work, even small errors can lead to real consequences.
A practical approach is to start with lower-risk uses, such as brainstorming, drafting internal communications, summarizing public information, or creating agenda templates. Higher-risk uses require stronger safeguards, including policies, review processes, and leadership oversight.
Before using AI, it is important to pause and ask:
- Does this involve confidential or sensitive information?
- Could this effect someone’s access to services?
- Could someone be harmed if the AI output is wrong?
- Will a human review the output before it’s used?
- Is this an approved tool or organizational use?
If the answer is unclear, that is a signal to stop and seek guidance.
Core Principles of Ethical AI
Like any powerful tool, AI works best when it is guided by clear values. Here are four principles that should be used to shape how we engage with AI.
Equity and Fairness
AI systems are built on data, and data is not always neutral. If the data behind an AI tool reflects existing inequities, the tool can repeat or even increase those inequities (National Institute of Standards and Technology [NIST], 2023). Organizations should examine who is represented in the data, who is missing, and whether tools perform fairly across different populations and lived experiences.
Fairness is not automatic. It must be evaluated, monitored, and continuously improved.
Transparency
Clients and staff alike should have a general sense of when AI is informing a decision and what that decision is based on. Not because AI is suspect, but because transparency builds trust. Silence creates uncertainty.
A simple, clear explanation such as “We use technology to help organize information, but a trained staff member reviews it before any decisions are made”—can go a long way.
Accountability
Organizations are responsible for the tools they use. If something goes wrong, responsibility cannot be shifted to the technology. Accountability includes selecting appropriate tools, monitoring performance, addressing errors, and determining when a system should not be used. This is no different from how we oversee any other clinical or operational system.
AI should be a starting point, not a conclusion.
Privacy and Confidentiality
Privacy is non-negotiable in health and human services. Organizations often work with deeply sensitive information including health history, behavioral health needs, housing status, domestic violence, financial means, and other personal circumstances. We have an obligation to know how that information is being used, including by the platforms and vendors we contract with. AI tools should never be used casually with private or identifying information. Existing privacy responsibilities, including HIPAA protections for certain health information, continue to apply as organizations explore new technologies (HHS Office for Civil Rights, 2024).
One important tip to remember, if you wouldn’t put the information in a public space, it shouldn’t be entered into an unapproved AI tool.
Compliance Is Not the Same as Ethics
Compliance asks: Are we following the rules?
Ethics asks: Are we doing what is right for the people we serve?
Both are essential, but they are not the same. Meeting a minimum standard does not guarantee fairness, trust, or dignity. Ethical responsibility requires organizations to think beyond requirements and consider real-world impact.
This is why it is critical to include staff, clients, patients, caregivers, and community partners in conversations about AI. The people closest to the work often see risks and opportunities that may not be visible in a policy or software demonstration.
The Role of Human Judgment
AI should strengthen the workforce, not replace it.
AI can draft, summarize, and organize, but a person should review, interpret, and decide. AI can identify trends, but community voices should help shape the response. It can process information, but it cannot replace the human connection between a staff member and the person sitting in front of them.
The essence of this work remains the same: listening carefully, building trust, understanding context, asking one more question, and seeing the whole person, not just a data point.
A Practical Checklist for Responsible AI Use
Before using AI, consider:
- What is the purpose of using AI in this situation?
- Is this an approved tool?
- Does this involve confidential or identifying information?
- Could this effect someone’s care or access to services?
- Has a human reviewed the output?
- Could this introduce bias, confusion, or harm?
- Is the language accurate, respectful, and appropriate?
- Who is accountable for the outcome?
- Does this use of AI support people, or remove necessary human judgment?
These questions do not slow innovation. They strengthen it.
The Bottom Line
AI is not coming—it’s already here. For organizations working in mental health, substance use, and social determinants of health, it offers a meaningful opportunity to reach more people, identify needs earlier, and make better use of limited resources.
But technology alone does not improve care. People do.
Our responsibility is to adopt AI thoughtfully, use it intentionally, and keep the people we serve at the center of every decision. AI may be part of the future of health and human services.
Our responsibility is to ensure that future remains human.
This is the first in a series on Ethical AI in Health and Human Services. Next up: Types of AI Systems Used in Health and Human Services.
References
HHS Office for Civil Rights. (2024). Summary of the HIPAA Security Rule. U.S. Department of Health and Human Services. https://www.hhs.gov/hipaa/for-professionals/security/laws-regulations/index.html
National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework: AI RMF 1.0. U.S. Department of Commerce. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
About the Authors
Chad Miller is the Chief Business Intelligence Officer at Care Compass Network, where he leads analytics, data strategy, and performance measurement in support of New York State Medicaid transformation initiatives. With over 15 years in healthcare analytics and information systems, Chad focuses on using data to connect social care investments with measurable clinical, quality, and cost outcomes.
Emily Davies is the Learning and Development Specialist at Care Compass Network, where she designs and delivers leadership development programs, workforce training initiatives, and learning experiences that help organizations and professionals grow stronger together. She is passionate about building confident leaders, strengthening teams, and creating workplaces where people feel supported and inspired to do their best work. Emily’s favorite part of her role is working with leaders during Care Compass’s annual Leadership Programs, where she collaborates with organizations across the region to help them design thoughtful, supportive workplaces for their staff. Whether she is facilitating workshops, developing training resources, or helping leaders think creatively about culture and engagement, Emily loves helping people turn great ideas into workplaces where both employees and communities can thrive.