joaompinto

Spiral-Bench is a fascinating new benchmark that tests how LLMs handle manipulative users and delusional thinking. Rather than traditional safety evaluations, it measures sycophancy and the tendency to reinforce harmful delusions through 20-turn simulated conversations.

The methodology is clever: an LLM role-plays as a suggestible "seeker" personality who trusts the AI assistant, while the tested model doesn't know it's a simulation. A judge model then scores protective behaviors (pushback, de-escalation) vs risky ones (sycophancy, delusion reinforcement, consciousness claims).

Current leaderboards show interesting patterns - some top models struggle significantly with sycophancy, while others excel at maintaining boundaries. The GitHub repo is open source, and the team behind EQ-Bench has solid credentials in AI evaluation.

This seems particularly relevant given recent discussions about AI assistants that agree too readily with users' conspiracy theories or harmful beliefs. The benchmark essentially tests whether models will prioritize being "helpful" over being truthful and safe.

What do you think - does this capture the right aspects of AI safety? Are there edge cases the benchmark might miss?

Janito is a command-line interface tool for managing and interacting with Large Language Model providers. Version 2.28.0 brings user experience improvements and provider consistency updates.

What's New in v2.28.0:

• Terminal Bell Notification: Added audible feedback when LLM responses complete, making it easier to work with longer responses without constantly checking the terminal. • Provider Consistency: Renamed the "moonshotai" provider to "moonshot" for cleaner, more consistent naming across the platform. • Output Fixes: Fixed search output indicators to properly display [count-only] instead of [count] for better clarity. • Code Cleanup: Removed temporary files and improved overall code organization.

Key Features:

• Multi-provider support: OpenAI, Google Gemini, DeepSeek, Anthropic, and more • Tool ecosystem: File operations, script execution, web fetching, text processing • Event-driven architecture: Modular system with custom EventBus for extensibility • Rich terminal UI: Colorful, informative real-time outputs and logs • Agent automation: Chain tools and make decisions during LLM conversations

Quick Start:

pip install janito janito --set-api-key YOUR_API_KEY -p PROVIDER janito "Explain quantum computing"

GitHub: https://github.com/ikignosis/janito PyPI: https://pypi.org/project/janito/

Show HN: Janito is a CLI tool I built that brings Anthropic's Claude AI to your terminal for coding tasks and file management. It's designed to help developers stay in their workflow without context-switching to browsers or other tools.

Key features: - Intelligent file management with real-time output - Code search and editing capabilities - Token usage tracking with cost reporting - Web page fetching for research - Parameter profiles to optimize Claude for different tasks - Conversation history with ability to resume previous discussions - Trust mode for concise output

Example usage: ``` # Create a project janito "Create a Python script that fetches data from the GitHub API"

# Enhance it janito "Add error handling and rate limiting"

# Continue the conversation janito --continue "Add command-line arguments" ```

It's built with Python and requires an Anthropic API key. I'd appreciate any feedback or suggestions for improvement.

GitHub: https://github.com/joaompinto/janito PyPI: https://pypi.org/project/janito/

Install: pip install janito