From 8000f52dfdbab97f640d94ed51debb63e4e79824 Mon Sep 17 00:00:00 2001 From: Daniel Ari Friedman Date: Fri, 7 Feb 2025 06:27:42 -0800 Subject: [PATCH] Updates --- README.md | 323 ++++++++++++++++++++++++++++++++++++++++++- project_structure.md | 146 +++++++++++++++++++ requirements.txt | 47 +++++++ 3 files changed, 515 insertions(+), 1 deletion(-) create mode 100644 project_structure.md create mode 100644 requirements.txt diff --git a/README.md b/README.md index 33110f3..07b0dac 100644 --- a/README.md +++ b/README.md @@ -1 +1,322 @@ -# cognitive \ No newline at end of file +# Cognitive Ecosystem Modeling Framework + +A comprehensive framework for modeling cognitive ecosystems using [[active_inference|Active Inference]], integrated with [[obsidian_linking|Obsidian]] for knowledge management. + +## Overview + +This project combines cognitive modeling with knowledge management to create a powerful framework for: +- Modeling agent behaviors using [[active_inference|Active Inference]] +- Managing complex [[knowledge_organization|knowledge structures]] +- Visualizing and analyzing [[cognitive_phenomena|cognitive networks]] +- Simulating multi-agent interactions + +## Project Structure + +See [[ai_folder_structure]] for comprehensive directory organization. + +``` +📁 templates/ # Template definitions +├── node_templates/ # Base templates for cognitive nodes +│ ├── agent_template.md # See [[ai_concept_template]] +│ ├── belief_template.md +│ └── ... +│ +📁 knowledge_base/ # Knowledge structure +├── cognitive/ # Core cognitive concepts +├── agents/ # Agent definitions +├── beliefs/ # Belief networks +└── ... + +📁 src/ # Source code +├── models/ # Core modeling components +├── utils/ # Utility functions +└── analysis/ # Analysis tools + +📁 docs/ # Documentation (See [[documentation_standards]]) +📁 tests/ # Test suite (See [[testing_guide]]) +📁 data/ # Data storage +``` + +## Features + +### Knowledge Management +- [[obsidian_linking|Obsidian-compatible markdown files]] +- [[linking_completeness|Bidirectional linking]] +- [[ai_concept_template|Template-based node creation]] +- [[ai_validation_framework|Automated relationship tracking]] + +### Cognitive Modeling +- [[active_inference|Active Inference implementation]] +- [[belief_updating|Belief updating mechanisms]] +- [[action_selection|Policy selection algorithms]] +- [[predictive_processing|State estimation tools]] + +### Analysis & Visualization +- [[network_analysis|Network analysis]] +- [[quality_metrics|Performance metrics]] +- [[visualization_tools|Interactive visualizations]] +- [[simulation_studies|Simulation frameworks]] + +## Knowledge Integration Architecture + +### Bidirectional Knowledge Graph +The framework leverages [[obsidian_linking|Obsidian's linking capabilities]] to create a living knowledge graph that: +- Enforces [[validation_framework|mathematical and theoretical consistency]] +- Enables [[ai_validation_framework|automated validation]] of relationships +- Supports [[machine_readability|dynamic discovery]] of dependencies +- Facilitates [[research_education|learning through exploration]] + +#### Link Types and Semantics +See [[linking_completeness]] for comprehensive linking patterns. + +1. Theoretical Dependencies + ```markdown + [[measure_theory]] → [[probability_theory]] → [[stochastic_processes]] + ``` + - Enforces prerequisite knowledge + - Validates theoretical foundations + - Ensures consistent notation + +2. Implementation Dependencies + ```markdown + [[active_inference]] → [[belief_updating]] → [[action_selection]] + ``` + - Tracks computational requirements + - Maintains implementation consistency + - Documents design decisions + +3. Validation Links + ```markdown + [[testing_guide]] → [[validation_framework]] → [[quality_metrics]] + ``` + - Ensures rigorous testing + - Maintains quality standards + - Documents validation procedures + +### Probabilistic Programming Integration + +#### Graph Structure Mapping +The repository's link structure directly maps to probabilistic graphical models: +```python +# Example: Converting knowledge links to Bayesian Graph +def build_bayesian_graph(knowledge_base: Path) -> BayesianNetwork: + """Convert knowledge base links to Bayesian Network. + See [[ai_semantic_processing]] for details. + """ + graph = BayesianNetwork() + + # Extract links and dependencies + for file in knowledge_base.glob('**/*.md'): + links = extract_links(file) + nodes = create_nodes(links) + edges = create_edges(links) + + # Add to graph with conditional probabilities + graph.add_nodes(nodes) + graph.add_edges(edges) + + return graph +``` + +#### Implementation Patterns +See [[package_documentation]] for detailed implementation guidelines. + +1. Direct Specification + ```yaml + # In matrix_specification.md + matrix: + type: observation + dimensions: [num_states, num_observations] + distribution: categorical + parameters: + prior: dirichlet + concentration: [1.0, ..., 1.0] + ``` + +2. Probabilistic Annotations + ```python + @probabilistic_model + class ObservationModel: + """Implementation with probabilistic annotations. + See [[predictive_processing]] for theoretical background. + """ + def __init__(self): + self.A = PyroMatrix(dims=['states', 'obs']) + + def forward(self, state): + return pyro.sample('obs', + dist.Categorical(self.A[state])) + ``` + +3. Inference Specifications + ```yaml + inference: + method: variational + guide: mean_field + optimizer: adam + parameters: + learning_rate: 0.01 + num_particles: 10 + ``` + +### Knowledge Base Integration + +#### Automated Validation +```python +def validate_knowledge_base(): + """Validate theoretical consistency. + See [[ai_validation_framework]] for details. + """ + # Check link consistency + validate_theoretical_dependencies() + + # Verify probabilistic specifications + validate_probability_constraints() + + # Test implementation coherence + validate_implementation_patterns() +``` + +#### Learning Pathways +See [[research_education]] for comprehensive learning integration. + +- **Theory**: Follow theoretical dependency chains + ```markdown + [[measure_theory]] → [[probability_theory]] → [[active_inference]] + ``` + +- **Implementation**: Track implementation requirements + ```markdown + [[matrix_design]] → [[numerical_methods]] → [[optimization]] + ``` + +- **Validation**: Ensure testing coverage + ```markdown + [[unit_tests]] → [[integration_tests]] → [[system_validation]] + ``` + +### Meta-Programming Capabilities + +#### Code Generation +```python +def generate_model_code(spec_file: Path) -> str: + """Generate implementation from specifications. + See [[ai_documentation_style]] for code generation patterns. + """ + # Parse markdown specifications + spec = parse_markdown_spec(spec_file) + + # Extract probabilistic model + model = extract_probabilistic_model(spec) + + # Generate implementation + return generate_implementation(model) +``` + +#### Validation Rules +```python +def check_probabilistic_consistency(): + """Verify probabilistic consistency. + See [[validation_framework]] for validation rules. + """ + # Check matrix constraints + verify_stochastic_matrices() + + # Validate probability measures + verify_measure_consistency() + + # Check inference specifications + verify_inference_methods() +``` + +### Benefits + +1. **Theoretical Consistency** + - [[ai_validation_framework|Automated validation]] of mathematical relationships + - Enforcement of probabilistic constraints + - Verification of implementation patterns + +2. **Learning Support** + - [[research_education|Guided exploration]] of concepts + - Clear dependency tracking + - Interactive knowledge discovery + +3. **Implementation Quality** + - [[ai_documentation_style|Automated code generation]] + - Consistent design patterns + - [[testing_guide|Rigorous testing framework]] + +4. **Documentation Integration** + - [[documentation_standards|Living documentation]] + - [[package_documentation|Executable specifications]] + - [[ai_validation_framework|Automated validation]] + +## Getting Started + +1. **Setup Environment** + ```bash + python -m venv venv + source venv/bin/activate # or `venv\Scripts\activate` on Windows + pip install -r requirements.txt + ``` + +2. **Configure Project** + - Edit `config.yaml` for project settings + - Customize templates in `templates/` + - Set up [[obsidian_linking|Obsidian vault integration]] + +3. **Create Cognitive Models** + - Use [[ai_concept_template|templates]] to define agents + - Configure belief networks + - Set up observation spaces + - Define action policies + +4. **Run Simulations** + - Execute model simulations + - Analyze results + - Visualize networks + +## Testing + +See [[testing_guide]] for comprehensive testing documentation. + +### Running Tests +```bash +# Run all tests with verbose output +python -m pytest -v + +# Run specific test file +python -m pytest tests/test_matrix_ops.py -v + +# Run tests with coverage report +python -m pytest --cov=src +``` + +### Test Organization +- `tests/test_matrix_ops.py`: Matrix operation tests +- `tests/test_visualization.py`: Visualization component tests +- `tests/conftest.py`: Shared test fixtures and configuration + +## Development + +### Contributing +See [[contribution_guide]] for detailed contribution guidelines. + +### Documentation +- [[api_documentation|API Documentation]] +- [[documentation_guide|User Guide]] +- [[example_writing|Examples]] + +## License + +This project is licensed under the MIT License - see the LICENSE file for details. + +## Acknowledgments + +- Active Inference research community +- Obsidian development team +- Contributors and maintainers + + + + diff --git a/project_structure.md b/project_structure.md new file mode 100644 index 0000000..0e52263 --- /dev/null +++ b/project_structure.md @@ -0,0 +1,146 @@ +# Project Structure + +## Core Directories + +### 📁 templates/ +- `node_templates/` - Base templates for different types of cognitive nodes (See [[ai_concept_template]]) + - `agent_template.md` - Template for [[active_inference]] agent definitions + - `belief_template.md` - Template for [[belief_updating]] structures + - `goal_template.md` - Template for goal hierarchies and optimization + - `action_template.md` - Template for [[action_selection]] definitions + - `observation_template.md` - Template for [[predictive_processing]] patterns + - `relationship_template.md` - Template for [[linking_completeness|node relationships]] + +### 📁 knowledge_base/ +- `cognitive/` - Core cognitive concepts (See [[cognitive_phenomena]]) + - [[active_inference]] - Active Inference framework + - [[free_energy_principle]] - Free Energy Principle + - [[predictive_processing]] - Predictive Processing theory +- `agents/` - Agent definitions and states +- `beliefs/` - Belief networks and structures +- `goals/` - Goal hierarchies and definitions +- `actions/` - Action repertoires and policies +- `observations/` - Observation patterns and histories +- `relationships/` - Inter-node relationships and dynamics + +### 📁 src/ +- `models/` - Core modeling components + - `active_inference/` - [[active_inference|Active Inference]] implementation + - `belief_updating/` - [[belief_updating|Belief update mechanisms]] + - `policy_selection/` - [[action_selection|Policy selection algorithms]] + - `state_estimation/` - State estimation tools +- `utils/` - Utility functions and helpers + - `visualization/` - [[visualization_tools|Visualization tools]] + - `data_processing/` - Data processing utilities + - `obsidian_integration/` - [[obsidian_linking|Obsidian integration tools]] +- `analysis/` - Analysis tools and scripts + - `network_analysis/` - Network analysis tools + - `metrics/` - Performance and behavior metrics + - `simulations/` - Simulation frameworks + +### 📁 docs/ +See [[documentation_standards]] for comprehensive documentation guidelines + +#### Core Documentation +- `api/` - API Reference Documentation + - [[api_documentation]] - API documentation guidelines + - [[api_reference]] - Complete API reference + - [[api_versioning]] - API versioning guidelines + - [[api_examples]] - API usage examples + +#### Conceptual Documentation +- `concepts/` - Core Concepts and Theory + - [[machine_readability]] - Machine processing principles + - [[plain_text_benefits]] - Benefits of plain text + - [[research_education]] - Research and education integration + - [[cognitive_phenomena]] - Cognitive phenomena catalog + - [[theoretical_foundations]] - Theoretical background + +#### Implementation Guides +- `guides/` - Usage Guides and Tutorials + - [[ai_documentation_style]] - Documentation style guide + - [[ai_file_organization]] - File organization guide + - [[ai_folder_structure]] - Directory structure guide + - [[ai_semantic_processing]] - Semantic processing guide + - [[ai_validation_framework]] - Validation framework + - [[linking_completeness]] - Link completeness guide + - [[linking_analysis]] - Link analysis guide + - [[linking_validation]] - Link validation guide + - [[obsidian_linking]] - Obsidian integration guide + - [[package_documentation]] - Package documentation guide + +#### Example Documentation +- `examples/` - Example Implementations + - [[example_writing]] - Example writing guide + - [[quickstart_example]] - Quick start tutorial + - [[active_inference_example]] - Active inference examples + - [[belief_updating_example]] - Belief updating examples + - [[integration_examples]] - Integration examples + - [[validation_examples]] - Validation examples + +#### Template Documentation +- `templates/` - Documentation Templates + - [[package_component]] - Package component template + - [[ai_concept_template]] - AI concept template + - [[linking_template]] - Linking template + - [[validation_template]] - Validation template + - [[example_template]] - Example template + - [[guide_template]] - Guide template + +#### Tool Documentation +- `tools/` - Development Tools + - [[documentation_tools]] - Documentation tooling + - [[validation_tools]] - Validation tools + - [[visualization_tools]] - Visualization tools + - [[analysis_tools]] - Analysis tools + - [[generation_tools]] - Code generation tools + +#### Research Documentation +- `research/` - Research Documentation + - [[experiment_documentation]] - Experiment documentation + - [[result_analysis]] - Result analysis + - [[literature_review]] - Literature review + - [[theoretical_development]] - Theory development + +### 📁 tests/ +- Unit tests and integration tests (See [[testing_guide]]) + - `test_matrix_ops.py` - Matrix operation tests + - `test_visualization.py` - Visualization tests + - `conftest.py` - Shared test fixtures + +### 📁 data/ +- `raw/` - Raw data storage +- `processed/` - Processed data +- `results/` - Analysis results + +## Key Files + +- `requirements.txt` - Python dependencies +- `setup.py` - Package setup +- `README.md` - Project documentation (See [[documentation_standards]]) +- `CONTRIBUTING.md` - [[contribution_guide|Contribution guidelines]] +- `.gitignore` - [[git_workflow|Git ignore rules]] +- `config.yaml` - Configuration settings + +## Documentation Integration + +### Knowledge Graph Structure +The project follows a [[linking_completeness|comprehensive linking structure]] that ensures: +- Bidirectional relationships between components +- Clear dependency chains +- Traceable implementations +- Validated connections + +### Validation Framework +See [[ai_validation_framework]] for details on: +- Link validation +- Documentation completeness +- Quality metrics +- Automated checks + +### Best Practices +Follow these guides for development: +- [[ai_documentation_style]] - Documentation style guide +- [[ai_file_organization]] - File organization patterns +- [[ai_folder_structure]] - Directory structure guidelines +- [[ai_semantic_processing]] - Semantic processing integration \ No newline at end of file diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..5447c9b --- /dev/null +++ b/requirements.txt @@ -0,0 +1,47 @@ +# Core Scientific Computing +numpy>=1.21.0 +scipy>=1.7.0 +pandas>=1.3.0 +scikit-learn>=0.24.0 + +# Deep Learning (Optional) +torch>=1.9.0 +tensorflow>=2.6.0 + +# Probabilistic Programming +pymc3>=3.11.0 +pomegranate>=0.14.0 + +# Visualization +matplotlib>=3.4.0 +seaborn>=0.11.0 +networkx>=2.6.0 +plotly>=5.1.0 + +# Data Processing +pyyaml>=5.4.0 +jinja2>=3.0.0 +markdown>=3.3.0 + +# Testing and Development +pytest>=6.2.0 +black>=21.6b0 +flake8>=3.9.0 +mypy>=0.910 + +# Documentation +sphinx>=4.0.0 +sphinx-rtd-theme>=0.5.0 + +# Utilities +tqdm>=4.61.0 +click>=8.0.0 +python-dotenv>=0.19.0 + +# Optimization +optuna>=2.9.0 +hyperopt>=0.2.5 + +# Monitoring and Logging +wandb>=0.12.0 +mlflow>=1.19.0 \ No newline at end of file