{"id":3391,"date":"2024-12-13T09:26:21","date_gmt":"2024-12-13T09:26:21","guid":{"rendered":"https:\/\/www.batteriesfast.co.uk\/blogs\/?p=3391"},"modified":"2024-12-13T09:26:22","modified_gmt":"2024-12-13T09:26:22","slug":"clio-a-privacy-preserving-system-that-provides-insights-into-real-world-ai-usage","status":"publish","type":"post","link":"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391","title":{"rendered":"Clio: A privacy-preserving system that provides insights into real-world AI usage"},"content":{"rendered":"\n

What are people using AI<\/strong> models for? Despite the rapid growth in popularity of large language models, we still know very little about what they are used for.<\/p>\n\n\n\n

This isn\u2019t just out of curiosity, or even sociological research. Understanding how people actually use language models is important for security reasons: providers put a lot of effort into pre-deployment testing and use trust and safety systems to prevent misuse. But the scale and diversity of language models makes it hard to understand what they are used for (not to mention any kind of comprehensive security monitoring).<\/p>\n\n\n\n

There\u2019s another key factor that prevents us from having a clear understanding of how  AI<\/strong> models are used: privacy. At Anthropic, our Claude model is not trained on user conversations by default, and we take protecting our users\u2019 data very seriously. So how do we study and observe how our systems are used while strictly protecting our users\u2019 privacy?<\/p>\n\n\n\n

Cl aude Insights and Observations (\u201cClio\u201d for short) is our attempt to answer this question. Clio is an automated analytics tool that performs privacy-preserving analysis of real-world language model usage. It gives us insight into everyday usage at claude.ai in a similar way to tools like Google Trends. It\u2019s already helping us improve our security measures. In this post (with the full research paper attached), we describe Clio and some of its initial results.<\/p>\n\n\n\n

How Clio Works: Privacy-Preserving Analytics at Scale<\/p>\n\n\n\n

Traditional top-down security approaches (such as assessments and red teams) rely on knowing what to look for in advance. Clio takes a different approach, enabling bottom-up pattern discovery by distilling conversations into abstract, understandable clusters of topics. It does this while protecting user privacy: data is automatically anonymized and aggregated, and only higher-level clusters are visible to human analysts.<\/p>\n\n\n\n

All of our privacy protections are extensively tested, as described in our research paper.<\/p>\n\n\n\n

How People Use Claude: Insights from Clio<\/p>\n\n\n\n

Using Clio, we were able to gain insight into how people use claude.ai in practice. While public datasets such as WildChat and LMSYS-Chat-1M provide useful information about how people use language models, they only capture specific contexts and use cases. Clio gives us a glimpse into the full real-world usage of claude.ai (which may differ from other  AI<\/strong> systems due to differences in user base and model type).<\/p>\n\n\n\n

Summary of Clio\u2019s analysis steps, illustrated using fictional examples of conversations.<\/p>\n\n\n\n

Here\u2019s a brief overview of Clio\u2019s multi-stage process:<\/p>\n\n\n\n

Extracting Aspects: For each conversation, Clio extracts multiple \u201caspects\u201d \u2014 specific properties or metadata, such as the topic of the conversation, the number of back-and-forths in the conversation, or the language used.<\/p>\n\n\n\n

Semantic Clustering: Similar conversations are automatically grouped based on topics or general themes.<\/p>\n\n\n\n

Cluster Descriptions: Each cluster receives a descriptive title and summary that captures common themes from the raw data while excluding private information.<\/p>\n\n\n\n

Building Hierarchies: Clusters are organized into multi-level hierarchies for easier exploration. They can then be presented in an interactive interface that a human factors analyst can use to explore patterns along different dimensions (topics, language, etc.).<\/p>\n\n\n\n

“Laptop Battery Online Shop- Replacement Laptop Battery, Cell phone Battery, Tablet Battery, Power Tools Battery Wholesale and Retailer in United Kingdom. All products are brand new with one year warranty and factory price!”<\/a><\/u><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

What are people using AI models for? Despite the rapid growth in popularity of large language models, we still know very little about what they are used for. This isn\u2019t just out of curiosity, or even sociological research. Understanding how people actually use language models is important for security reasons: providers put a lot of effort into […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3391","post","type-post","status-publish","format-standard","hentry","category-news"],"yoast_head":"\nClio: A privacy-preserving system that provides insights into real-world AI usage - Batteries Fast Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Clio: A privacy-preserving system that provides insights into real-world AI usage - Batteries Fast Blog\" \/>\n<meta property=\"og:description\" content=\"What are people using AI models for? Despite the rapid growth in popularity of large language models, we still know very little about what they are used for. This isn\u2019t just out of curiosity, or even sociological research. Understanding how people actually use language models is important for security reasons: providers put a lot of effort into […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391\" \/>\n<meta property=\"og:site_name\" content=\"Batteries Fast Blog\" \/>\n<meta property=\"article:published_time\" content=\"2024-12-13T09:26:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-12-13T09:26:22+00:00\" \/>\n<meta name=\"author\" content=\"bella\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"bella\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391\",\"url\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391\",\"name\":\"Clio: A privacy-preserving system that provides insights into real-world AI usage - Batteries Fast Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/#website\"},\"datePublished\":\"2024-12-13T09:26:21+00:00\",\"dateModified\":\"2024-12-13T09:26:22+00:00\",\"author\":{\"@id\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/#\/schema\/person\/77c84eb3fb06925fe618ddb29d55932b\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.batteriesfast.co.uk\/blogs\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Clio: A privacy-preserving system that provides insights into real-world AI usage\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/#website\",\"url\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/\",\"name\":\"Batteries Fast Blog\",\"description\":\"Replacement Battery & Adapter Blog\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/#\/schema\/person\/77c84eb3fb06925fe618ddb29d55932b\",\"name\":\"bella\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/bebf3096a51ce7e2123b6e0fc60d24a5e67ee865894d2a376f0cd59eb487b2fa?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/bebf3096a51ce7e2123b6e0fc60d24a5e67ee865894d2a376f0cd59eb487b2fa?s=96&d=mm&r=g\",\"caption\":\"bella\"},\"sameAs\":[\"https:\/\/batteriesfast.co.uk\/blogs\"],\"url\":\"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/author\/bella\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Clio: A privacy-preserving system that provides insights into real-world AI usage - Batteries Fast Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391","og_locale":"en_US","og_type":"article","og_title":"Clio: A privacy-preserving system that provides insights into real-world AI usage - Batteries Fast Blog","og_description":"What are people using AI models for? Despite the rapid growth in popularity of large language models, we still know very little about what they are used for. This isn\u2019t just out of curiosity, or even sociological research. Understanding how people actually use language models is important for security reasons: providers put a lot of effort into […]","og_url":"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391","og_site_name":"Batteries Fast Blog","article_published_time":"2024-12-13T09:26:21+00:00","article_modified_time":"2024-12-13T09:26:22+00:00","author":"bella","twitter_card":"summary_large_image","twitter_misc":{"Written by":"bella","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391","url":"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391","name":"Clio: A privacy-preserving system that provides insights into real-world AI usage - Batteries Fast Blog","isPartOf":{"@id":"https:\/\/www.batteriesfast.co.uk\/blogs\/#website"},"datePublished":"2024-12-13T09:26:21+00:00","dateModified":"2024-12-13T09:26:22+00:00","author":{"@id":"https:\/\/www.batteriesfast.co.uk\/blogs\/#\/schema\/person\/77c84eb3fb06925fe618ddb29d55932b"},"breadcrumb":{"@id":"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/3391#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.batteriesfast.co.uk\/blogs"},{"@type":"ListItem","position":2,"name":"Clio: A privacy-preserving system that provides insights into real-world AI usage"}]},{"@type":"WebSite","@id":"https:\/\/www.batteriesfast.co.uk\/blogs\/#website","url":"https:\/\/www.batteriesfast.co.uk\/blogs\/","name":"Batteries Fast Blog","description":"Replacement Battery & Adapter Blog","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.batteriesfast.co.uk\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.batteriesfast.co.uk\/blogs\/#\/schema\/person\/77c84eb3fb06925fe618ddb29d55932b","name":"bella","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.batteriesfast.co.uk\/blogs\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/bebf3096a51ce7e2123b6e0fc60d24a5e67ee865894d2a376f0cd59eb487b2fa?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/bebf3096a51ce7e2123b6e0fc60d24a5e67ee865894d2a376f0cd59eb487b2fa?s=96&d=mm&r=g","caption":"bella"},"sameAs":["https:\/\/batteriesfast.co.uk\/blogs"],"url":"https:\/\/www.batteriesfast.co.uk\/blogs\/archives\/author\/bella"}]}},"_links":{"self":[{"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/posts\/3391","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/comments?post=3391"}],"version-history":[{"count":1,"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/posts\/3391\/revisions"}],"predecessor-version":[{"id":3392,"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/posts\/3391\/revisions\/3392"}],"wp:attachment":[{"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/media?parent=3391"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/categories?post=3391"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.batteriesfast.co.uk\/blogs\/wp-json\/wp\/v2\/tags?post=3391"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}