loco-non-backprop-snn-learning

name: loco-non-backprop-snn-learning description: "LOCO (Low-rank Cluster Orthogonal) weight modification for backpropagation-free SNN training. Perturbation-based non-BP learning with O(1) parallel time complexity, enabling deep SNN training (10+ layers) with continual learning capability. Activation: non-backpropagation, LOCO, node perturbation, orthogonal weight, brain-inspired learning, neuromorphic training."

LOCO Non-Backpropagation SNN Learning

Perturbation-based orthogonal weight modification method for training deep Spiking Neural Networks without gradient backpropagation, achieving O(1) parallel time complexity and strong continual learning ability.

Metadata

• Source: arXiv:2602.22259
• Authors: Guoqing Ma, Shan Yu
• Published: 2026-02-25

Core Methodology

Key Innovation

LOCO (Low-rank Cluster Orthogonal) weight modification is a perturbation-based non-backpropagation learning algorithm that addresses efficiency and scalability challenges in existing non-BP approaches for neuromorphic systems.

Technical Framework

1. Low-Rank Property: LOCO exploits the finding that low-rank is an inherent property of perturbation-based algorithms
2. Orthogonality Constraint: Constrains weight updates to be orthogonal, which:
   • Limits variance of node perturbation (NP) gradient estimates
   • Enhances convergence efficiency
3. Cluster-based Organization: Groups neurons into clusters for localized weight modification
4. O(1) Parallel Time Complexity: Weight updates require constant parallel time, significantly lower than BP methods

Algorithm Steps

1. Initialize SNN with orthogonal weight clusters
2. Forward pass to compute network output
3. Apply node perturbation with orthogonal constraints
4. Compute local weight modifications within clusters
5. Update weights using low-rank orthogonal modification
6. Repeat for training iterations

Implementation Guide

Prerequisites

• Spiking Neural Network framework (SpikingJelly, Norse, or custom)
• Understanding of node perturbation algorithms
• Neuromorphic hardware target (optional)

Key Parameters

• Cluster size: Number of neurons per orthogonal cluster
• Perturbation magnitude: Scale of node perturbation
• Learning rate: Step size for weight modification
• Orthogonal constraint strength: Regularization for orthogonality

Code Example

import numpy as np

class LOCOLayer:
    """LOCO weight modification layer for non-BP SNN training."""
    
    def __init__(self, in_dim, out_dim, cluster_size=64):
        self.cluster_size = cluster_size
        # Initialize orthogonal weight clusters
        n_clusters = out_dim // cluster_size
        self.weights = np.zeros((out_dim, in_dim))
        for i in range(n_clusters):
            start = i * cluster_size
            end = (i + 1) * cluster_size
            W_cluster = np.random.randn(cluster_size, in_dim)
            Q, _ = np.linalg.qr(W_cluster.T)
            self.weights[start:end] = Q.T
        
    def perturb_and_update(self, x, output, target, lr=0.01, perturbation_std=0.1):
        """Apply LOCO weight modification without backprop."""
        # Node perturbation
        perturbation = np.random.normal(0, perturbation_std, output.shape)
        perturbed_output = output + perturbation
        
        # Compute loss difference
        loss_diff = self._compute_loss(perturbed_output, target) - \
                    self._compute_loss(output, target)
        
        # Orthogonal weight update (low-rank)
        gradient_estimate = loss_diff * perturbation / (perturbation_std ** 2)
        
        # Apply cluster-structured update
        n_clusters = output.shape[0] // self.cluster_size
        for i in range(n_clusters):
            start = i * self.cluster_size
            end = (i + 1) * self.cluster_size
            delta = lr * np.outer(gradient_estimate[start:end], x)
            # Project onto orthogonal subspace
            W_cluster = self.weights[start:end]
            delta_proj = delta - W_cluster @ W_cluster.T @ delta
            self.weights[start:end] += delta_proj

Applications

• Deep SNN training: Train SNNs with 10+ layers without backpropagation
• Neuromorphic systems: Deploy on hardware that doesn't support BP
• Continual learning: Strong anti-forgetting properties due to localized updates
• Real-time learning: O(1) parallel time complexity enables online adaptation
• Lifelong learning systems: Combine with other plasticity rules for sustained learning

Advantages over Existing Non-BP Methods

1. Depth: Can train deeper SNNs than existing non-BP approaches
2. Efficiency: O(1) parallel time vs. O(N) for standard node perturbation
3. Convergence: Orthogonality constraint reduces gradient estimate variance
4. Continual learning: Inherently better at avoiding catastrophic forgetting

Pitfalls

• Perturbation sensitivity: Requires careful tuning of perturbation magnitude
• Cluster size trade-off: Too small clusters limit expressivity; too large reduce efficiency
• Not universal: May not match BP performance on all tasks
• Spiking-specific: Requires adaptation for specific neuron models (LIF, Izhikevich, etc.)

Related Skills

• snn-learning-survey
• multi-plasticity-snn-training
• decolle-snn-learning
• three-factor-snn-learning
• bio-neuron-snn-learning
• neuromodulated-synaptic-plasticity
• quantized-snn-hardware-optimization