hack/tools/scripts/llm-key-hunter/results/_cache/content/angelo-casciani_conversational-ocel2.json

{"repo": "angelo-casciani/conversational-ocel2", "files": {"README.md": {"99901993268ecf6a9b9ab2b0aa538982130e53c1": {"t": 1786068585.5020432, "n": 9882, "h": "5a6bfb5d904f7f48", "c": "# Enabling natural language analysis for object-centric event logs\r\n\r\n[![Sapienza](https://img.shields.io/badge/Sapienza_University_of_Rome-blue?style=flatlabelColor=781A2D&color=781A2D&logoColor=white)](https://www.diag.uniroma1.it/gruppi-di-ricerca/25462)\r\n[![DOI](https://img.shields.io/badge/DOI-/zenodo.21807813-blue.svg)](https://doi.org/10.5281/zenodo.21807813)\r\n[![Version](https://img.shields.io/badge/release-v1.0.1-blue)](https://github.com/angelo-casciani/Conversational-OCEL2/releases)\r\n[![GitHub license](https://img.shields.io/github/license/angelo-casciani/Conversational-OCEL2)](https://github.com/angelo-casciani/Conversational-OCEL2/blob/main/LICENSE)\r\n\r\nThis repository contains the code and data to reproduce the experiments from the paper [Enabling natural language analysis for object-centric event logs](https://link.springer.com/article/10.1007/s44311-026-00037-9).\r\nConversational-OCEL2 is a conversational framework designed to facilitate process mining analysis over object-centric event logs following the OCEL 2.0 standard (in JSON). \r\nThe approach leverages an architecture that combines Large Language Models (LLMs) with Retrieval Augmented Generation (RAG) to handle users' queries about OCEL 2.0 event logs and generate contextually relevant responses in natural language.\r\n\r\nAdditionally, the repository contains a dataset for evaluating the conversational framework, derived from a standard [OCEL 2.0 Procure-to-Pay (P2P) event log](https://www.ocel-standard.org/event-logs/simulations/p2p/).\r\nThis dataset functions as a benchmark for evaluating the effectiveness of conversational techniques in analyzing such event log from multiple perspectives.\r\n\r\n## Structure of the repository\r\n\r\n```\r\n.\r\n├── data/\r\n│   ├── execution              # Knowledge extracted from the event log\r\n│   └── ocel2-p2p.json         # Event log used for the evaluation\r\n├── src/ \r\n│   ├── cmd4tests.sh           # Commands for launching evaluations\r\n│   ├── eval.py                # Logic for Evaluation\r\n│   ├── main.py                # Main logic for live interaction\r\n│   ├── oracle.py              # Verification oracle for evaluation\r\n│   ├── pipeline.py            # LLM pipeline setup\r\n│   ├── preprocessing.py       # OCEL2 log preprocessing\r\n│   ├── prompts.json           # LLM prompt templates\r\n│   ├── utility.py             # Helper functions\r\n│   └── vector_store.py        # Vector store management with Qdrant\r\n├── tests/                     # sources for evaluation\r\n│   ├── outputs/               # outputs of the live conversations\r\n│   ├── test_sets/             # test sets employed during the evaluation\r\n│   └── validation/            # evaluation results for each run\r\n├── logs.zip                   # zipped folder with the tested log (to unzip)\r\n├── .env                       # Environment variables (create/fill this)\r\n├── requirements.txt           # Requirements to install\r\n├── LICENSE                    # License file\r\n└── README.md                  # This file\r\n```\r\n\r\n## Getting Started\r\n\r\nFor a quick setup and test run:\r\n\r\n1. **Clone and setup:**\r\n```bash\r\ngit clone https://github.com/angelo-casciani/Conversational-OCEL2\r\ncd Conversational-OCEL2\r\n```\r\n\r\nCreate a Python virtual environment.\r\n\r\n**Option 1: Using venv**\r\n```bash\r\npython3 -m venv .venv\r\nsource .venv/bin/activate\r\n```\r\n\r\n**Option 2: Using conda**\r\n```bash\r\nconda create --name xes2pddl python=3.10\r\nconda activate xes2pddl\r\n```\r\n\r\nInstall the required dependencies.\r\n\r\n```bash\r\npip install -r requirements.txt\r\n```\r\n\r\n2. **Start Qdrant (vector database):**\r\nThis project uses [Docker](https://www.docker.com/) to run the vector store [Qdrant](https://qdrant.tech/).\r\n\r\nEnsure Docker is installed and running on your system.\r\n\r\nDownload the latest Qdrant image from Docker Hub and  run the Qdrant service:\r\n```bash\r\ndocker pull qdrant/qdrant\r\ndocker run -p 6333:6333 -p 6334:6334 \\\r\n    -v \"$(pwd)/qdrant_storage:/qdrant/storage:z\" \\\r\n    qdrant/qdrant\r\n```\r\n\r\n3. **Configure environment (create `.env` file):**\r\n\r\nCreate a `.env` file in the root directory and configure the following variables:\r\n\r\n```env\r\nHF_TOKEN=<your HuggingFace token>\r\nDEEPSEEK_API_KEY=<your DeepSeek API key (if using DeepSeek models)>\r\nGOOGLE_API_KEY=<your Gemini API key (if using Google models)>\r\nOPENAI_API_KEY=<your OpenAI API key (if using OpenAI models)>\r\nQDRANT_URL=127.0.0.0\r\nQDRANT_GRPC_PORT=6334\r\n```\r\n\r\n**Required configurations:**\r\n- **HF_TOKEN**: Your HuggingFace token for accessing open-source language models and embedding models\r\n- **QDRANT_URL**: URL where Qdrant is running (default: `QDRANT_URL=127.0.0.0`)\r\n- **QDRANT_GRPC_PORT**: gRPC port for Qdrant (default: `6334`)\r\n\r\nThe other configurations are optional.\r\n\r\n\r\n4. **Run the application:**\r\n```bash\r\ncd src\r\npython3 main.py --rebuild_db True\r\n```\r\n\r\n## LLMs Requirements\r\n\r\nPlease note that this software leverages open-source LLMs reported in the table:\r\n\r\n| Model | HuggingFace Link |\r\n|-----------|-----------|\r\n| meta-llama/Meta-Llama-3-8B-Instruct | [HF link](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) |\r\n| meta-llama/Meta-Llama-3.1-8B-Instruct | [HF link](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) |\r\n| meta-llama/Llama-3.2-1B-Instruct | [HF Link](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)|\r\n| meta-llama/Llama-3.2-3B-Instruct | [HF link](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) |\r\n| mistralai/Mistral-7B-Instruct-v0.2 | [HF link](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) |\r\n| mistralai/Mistral-7B-Instruct-v0.3 | [HF link](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) |\r\n| mistralai/Mistral-Nemo-Instruct-2407 | [HF link](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) |\r\n| mistralai/Ministral-8B-Instruct-2410 | [HF link](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410) |\r\n| Qwen/Qwen2.5-7B-Instruct | [HF link](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |\r\n| google/gemma-2-9b-it | [HF link](https://huggingface.co/google/gemma-2-9b-it) |\r\n| gpt-4o-mini | [OpenAI link](https://platform.openai.com/docs/models) |\r\n\r\nRequest in advance the permission to use each Llama model for your HuggingFace account.\r\nRetrive your OpenAI API key to use the supported GPT model.\r\n\r\nPlease note that each of the selected models have specific requirements in terms of GPU availability.\r\nIt is recommended to have access to a GPU-enabled environment meeting at least the minimum requirements for these models to run the software effectively.\r\n\r\n## Running the Project\r\n\r\n### Basic Usage\r\n\r\nNavigate to the project directory and run the project in the preferred configuration:\r\n\r\n```bash\r\ncd src\r\npython3 main.py\r\n```\r\n\r\n### Enhanced Pipeline (Recommended)\r\n\r\nThe project now includes an enhanced pipeline with better error handling and performance. To use it:\r\n\r\n```bash\r\npython3 main.py --modality=live\r\n```\r\n\r\n### Evaluation Modes\r\n\r\nTo run evaluations for different aspects of the OCEL2 analysis:\r\n\r\n**Global information evaluation:**\r\n```bash\r\npython3 main.py --llm_id Qwen/Qwen2.5-7B-Instruct --modality evaluation-global --max_new_tokens 512\r\n```\r\n\r\n**Events analysis evaluation:**\r\n```bash\r\npython3 main.py --llm_id Qwen/Qwen2.5-7B-Instruct --modality evaluation-events --max_new_tokens 512\r\n```\r\n\r\n**Objects analysis evaluation:**\r\n```bash\r\npython3 main.py --llm_id Qwen/Qwen2.5-7B-Instruct --modality evaluation-objects --max_new_tokens 512\r\n```\r\n\r\n**Timestamps analysis evaluation:**\r\n```bash\r\npython3 main.py --llm_id Qwen/Qwen2.5-7B-Instruct --modality evaluation-ts --max_new_tokens 512\r\n```\r\n\r\n**Complete evaluation (all categories):**\r\n```bash\r\npython3 main.py --llm_id Qwen/Qwen2.5-7B-Instruct --modality evaluation-all --max_new_tokens 512\r\n```\r\n\r\n### Database Rebuilding\r\n\r\nIf you need to rebuild the vector database (e.g., after changing the OCEL2 log or updating embeddings):\r\n\r\n```bash\r\npython3 main.py --rebuild_db=true \r\n```\r\n\r\n### Configuration Parameters\r\n\r\nThe framework supports various configuration parameters:\r\n\r\n| Parameter | Default | Description |\r\n|-----------|---------|-------------|\r\n| `--embed_model_id` | `sentence-transformers/all-MiniLM-L12-v2` | Embedding model identifier |\r\n| `--vector_dimension` | `384` | Vector space dimension (auto-detected if using enhanced pipeline) |\r\n| `--llm_id` | `meta-llama/Meta-Llama-3.1-8B-Instruct` | LLM model identifier |\r\n| `--model_max_length` | `128000` | Maximum input length (context window) |\r\n| `--num_documents_in_context` | `5` | Number of documents retrieved for context |\r\n| `--max_new_tokens` | `1280` | Maximum number of tokens to generate |\r\n| `--batch_size` | `32` | Batch size for embedding processing |\r\n| `--rebuild_db` | `false` | Whether to rebuild the vector index |\r\n| `--use_enhanced_pipeline` | `true` | Use enhanced pipeline with better error handling |\r\n\r\n### Custom OCEL2 Logs\r\n\r\nIt is possible to upload a different OCEL 2.0 log (in JSON format) in the *data* folder by replacing the provided *ocel2-p2p.json* log. After uploading a new log, rebuild the database:\r\n\r\n```bash\r\npython3 main.py --rebuild_db=true --log=your-new-log.json\r\n```\r\n\r\n### Usage with Enhanced Features\r\n```bash\r\n# Use enhanced pipeline (recommended)\r\npython3 main.py \r\n\r\n# Automatic embedding dimension detection\r\npython3 main.py --embed_model_id=sentence-transformers/all-mpnet-base-v2\r\n\r\n# Better error recovery and batch processing\r\npython3 main.py --rebuild_db=true --batch_size=16\r\n```\r\n\r\n## Citation\r\n\r\nIf you use this repository in your research, please cite:\r\n\r\n```bibtex\r\n@article{casciani2026enabling,\r\n  title={Enabling natural language analysis for object-centric event logs},\r\n  author={Casciani, Angelo and Bernardi, Mario Luca and Cimitile, Marta and Marrella, Andrea},\r\n  journal={Process Science},\r\n  volume={3},\r\n  number={1},\r\n  pages={5},\r\n  year={2026},\r\n  publisher={Springer}\r\n}\r\n```\r\n"}}}}