Technical Research

Research Papers

In-depth technical documentation of our fine-tuning methodology, training experiments, evaluation metrics, and lessons learned.

Educational AIJanuary 2025Zyora Labs Research Team

Efficient Domain Adaptation of Large Language Models for Educational Applications Using QLoRA

A technical report on fine-tuning Qwen2.5-Coder-32B-Instruct for syllabus-aligned educational assistance across engineering disciplines.

Abstract

We present Zyora-BYTE-32B, a domain-adapted large language model for educational applications. Using Quantized Low-Rank Adaptation (QLoRA), we fine-tuned the Qwen2.5-Coder-32B-Instruct base model on a curated dataset of 16,109 educational samples spanning multiple engineering disciplines. Our approach achieves a final training loss of 0.00844 after 3 epochs while requiring only 1.0 GB of additional adapter weights. This paper details our data curation process, hyperparameter selection, training dynamics, and deployment considerations for production educational AI systems.

1Introduction

Large language models have demonstrated remarkable capabilities in general-purpose tasks, but domain-specific applications often require specialized knowledge and response patterns. Educational AI presents unique challenges: models must align with specific curricula, provide pedagogically sound explanations, and adapt to varying student knowledge levels.

Full fine-tuning of 32B parameter models requires substantial computational resources—approximately 128GB of VRAM for training in full precision. This creates a significant barrier for organizations seeking to create specialized educational models. QLoRA (Quantized Low-Rank Adaptation) addresses this by enabling efficient fine-tuning through 4-bit quantization of the base model while training low-rank adapters in higher precision.

Our work focuses on creating an AI teaching assistant specifically designed for university-level engineering education. The model supports Computer Science, Electronics & Communication, Electrical, Mechanical, and Civil engineering curricula with syllabus-aligned responses.

2Methodology

2.1 Base Model Selection

We selected Qwen2.5-Coder-32B-Instruct as our base model for several reasons:

Strong performance on reasoning and explanation tasks
Native support for technical and mathematical content
Apache 2.0 license enabling commercial deployment
128K token context window for handling complex problems

2.2 Dataset Construction

Our training dataset comprises 16,109 samples constructed through:

Syllabus Extraction: University curricula from accredited engineering programs
Question-Answer Pairs: Faculty-verified Q&A covering core concepts
Problem-Solution Sets: Step-by-step solutions with explanations
Concept Explanations: Multi-level explanations from basic to advanced

Each sample follows a consistent format with clear instruction-response structure. We specifically avoided including copyrighted textbook content, instead focusing on faculty-created materials and publicly available educational resources.

2.3 QLoRA Configuration

Parameter	Value	Rationale
LoRA Rank (r)	32	Balance between capacity and efficiency
LoRA Alpha	64	Scaling factor (alpha/r = 2)
Dropout	0.05	Regularization to prevent overfitting
Learning Rate	2e-4	Standard for QLoRA fine-tuning
Batch Size	8 (effective)	Gradient accumulation for memory efficiency
Precision	bf16	Mixed precision training
Epochs	3	Based on validation loss monitoring

3Training Results

3.1 Training Dynamics

Training progressed smoothly over 3 epochs with consistent loss reduction:

~0.8

Initial Loss (Epoch 1 Start)

~0.15

Mid-training Loss (Epoch 2)

0.00844

Final Loss (Epoch 3 End)

3.2 Adapter Efficiency

The resulting LoRA adapter adds only 1.0 GB to the base model, representing approximately 3% additional parameters. This enables efficient deployment where the base model can be shared across multiple fine-tuned variants by simply swapping adapters.

Metric	Value
Base Model Size	~64 GB (FP16)
Adapter Size	1.0 GB
Total Training Samples	16,109
Training Steps	~6,000
Final Training Loss	0.00844

4Deployment Considerations

4.1 Hardware Requirements

Configuration	VRAM Required	Recommended Hardware
4-bit Quantized (GPTQ/AWQ)	~18 GB	RTX 4090, A5000
8-bit Quantized (bitsandbytes)	~32 GB	A100-40GB, 2x RTX 4090
Full Precision (FP16)	~64 GB	A100-80GB, H100

4.2 Inference Optimization

For production deployment, we recommend:

vLLM or TGI for efficient batched inference
KV-cache optimization for multi-turn conversations
Speculative decoding for reduced latency on long responses

5Limitations & Future Work

Known Limitations

Training data limited to English language content
Curriculum coverage may not include all university variants
Mathematical rendering requires proper frontend integration
Very low training loss (0.00844) may indicate some memorization—monitoring needed
No formal benchmarking against educational AI baselines yet conducted

Future directions include:

Multilingual support for regional language education
Integration with learning management systems
Adaptive difficulty based on student performance tracking
Formal evaluation on educational AI benchmarks

View Zyora-BYTE-32B Model Page

Code GenerationFebruary 2025Zyora Labs Research Team

Integrating Security Analysis into Code Generation: The Zyora-DEV-32B Approach

A technical report on building security-aware code generation with integrated vulnerability detection and remediation.

Abstract

Code generation models typically optimize for functional correctness without considering security implications. Zyora-DEV-32B addresses this gap by fine-tuning Qwen2.5-Coder-32B-Instruct with security-annotated training data that includes vulnerability patterns, OWASP Top 10 coverage, and remediation examples. The model generates code across 40+ programming languages while simultaneously identifying potential security issues and suggesting fixes. This paper describes our approach to creating security-conscious code generation through careful dataset construction and training methodology.

1Problem Statement

Modern code generation models excel at producing syntactically correct and functional code, but often introduce security vulnerabilities. Common issues include:

SQL injection through unsanitized user inputs
Cross-site scripting (XSS) in web application code
Hardcoded credentials and API keys
Insecure deserialization patterns
Path traversal vulnerabilities in file operations

Traditional approaches require separate static analysis tools post-generation, creating a fragmented workflow. Our goal was to integrate security awareness directly into the generation process.

2Technical Approach

2.1 Security-Annotated Dataset

We constructed a training dataset with three components:

Vulnerable Code Examples

Real-world vulnerable code snippets with CVE references and vulnerability type annotations.

Secure Alternatives

Corresponding secure implementations with explanations of the security improvements.

Remediation Pairs

Before/after examples showing how to fix specific vulnerability patterns.

2.2 OWASP Integration

Training data covers the OWASP Top 10 web application security risks:

Category	Training Focus
A01: Broken Access Control	Authorization checks, RBAC patterns
A02: Cryptographic Failures	Proper encryption usage, key management
A03: Injection	Parameterized queries, input sanitization
A04: Insecure Design	Security-first architecture patterns
A05-A10	Misconfiguration, components, authentication, integrity, logging

2.3 Multi-Language Support

The model supports 40+ programming languages, with security patterns adapted for each language's idioms and common frameworks. Primary languages include:

PythonJavaScriptTypeScriptJavaC++C#GoRustRubyPHPSwiftKotlin+28 more

3Output Format

When generating code, the model provides structured output including:

Generated Code: The requested implementation
Security Analysis: Identified potential issues (if any)
Risk Level: Severity classification of detected issues
Remediation: Suggested fixes with explanations

// Example Output Structure
{
  "code": "// Generated implementation...",
  "security_analysis": {
    "issues_found": 1,
    "issues": [
      {
        "type": "SQL_INJECTION",
        "severity": "HIGH",
        "line": 15,
        "description": "User input directly concatenated in SQL query",
        "cwe": "CWE-89",
        "remediation": "Use parameterized queries instead"
      }
    ]
  },
  "secure_alternative": "// Fixed implementation with parameterized query..."
}

4Limitations

Important Considerations

Security analysis is not a replacement for dedicated SAST/DAST tools
Cannot detect all vulnerability types, especially logic flaws
CVE database knowledge limited to training data cutoff date
Framework-specific vulnerabilities may not be fully covered
Should be used as one layer in a defense-in-depth security strategy

View Zyora-DEV-32B Model Page

AI Security2025Zyora Labs Research Team

Nexula-AIBOM-8B: Specialized Language Model for AI Bill of Materials and Security Analysis

A technical overview of our 8B parameter model designed for SBOM generation and AI supply chain security.

Abstract

As AI systems become critical infrastructure, understanding their components and potential vulnerabilities becomes essential. Nexula-AIBOM-8B is an 8 billion parameter model specifically designed for AI Bill of Materials (AIBOM) generation and security analysis. Built on a decoder-only transformer architecture with 48 layers and 128K token context window, the model was trained on a security-curated corpus and aligned using Constitutional AI with Direct Preference Optimization. This paper describes the model architecture, training approach, and target applications.

1Model Architecture

Component	Specification
Architecture	Decoder-only Transformer
Parameters	8 Billion
Layers	48
Attention Heads	32
Hidden Dimensions	4,096
Context Window	128,000 tokens
Vocabulary Size	100,000 tokens

2Training Approach

2.1 Dataset

The model was trained on a security-curated corpus with approximately 2 billion security-specific samples for fine-tuning, covering:

Software Bill of Materials (SBOM) in CycloneDX and SPDX formats
CVE and vulnerability databases
AI model card documentation
Supply chain security specifications
Adversarial attack patterns and mitigations

2.2 Alignment

Alignment was performed using a combination of Constitutional AI and Direct Preference Optimization (DPO) over 3 RLHF iterations. This approach ensures the model provides accurate security assessments while avoiding harmful outputs.

3Target Applications

SBOM Generation

Automated generation of Software Bill of Materials for AI models, including dependencies, training data sources, and model components.

Vulnerability Analysis

Scanning AI models and their supply chains for known vulnerabilities and potential security weaknesses.

Adversarial Detection

Identifying potential adversarial attack vectors and data poisoning indicators in AI systems.

Supply Chain Monitoring

Continuous monitoring of AI model supply chains for integrity and security compliance.

View Nexula-AIBOM-8B Model Page

Research Collaboration

Interested in collaborating on AI research, requesting model access, or discussing technical details? Get in touch with our team.

Contact Research Team View All Models