codenib.scip_interface ¶

Language-specific SCIP indexing and decoding.

The public classes stay lazy so importing an LSP-shaped provider does not load the optional igraph runtime or every language backend.

Modules:

Name	Description
`fact_batch_buffer`	Validated reader for the native `FactBatchBuffer v1` wire format.
`go_modules`	Go module layout for repositories split into nested modules.
`lsp_occurrence_index`	Exact position index for LSP-shaped navigation over decoded SCIP data.
`python_identity`	Recover physical Python declarations before either SCIP graph decoder runs.
`query_surface`	Canonical digest of the ordered symbol-query graph surface.
`rust_analyzer`	Helpers for invoking rust-analyzer consistently.
`scip_decode_core`	Full-graph Python facade around the low-level `codenib_core` extension.
`scip_decode_csharp`	SCIP decoder for C# projects indexed by scip-dotnet.
`scip_decode_go`	SCIP decoder for Go projects.
`scip_decode_java`	SCIP decoder for JVM projects indexed by scip-java.
`scip_decode_php`	SCIP decoder for PHP projects indexed by scip-php.
`scip_decode_python`	SCIP decoder for Python projects.
`scip_decode_ruby`	SCIP decoder for Ruby projects indexed by scip-ruby.
`scip_decode_rust`	SCIP decoder for Rust projects.
`scip_decode_ts`
`scip_decode_utils`	Shared normalization helpers for SCIP TextFormat decoders.
`scip_filter`	Dependency-free path filtering shared by SCIP graph and query routes.
`scip_indexer_base`	Base class for SCIP indexers across different languages.
`scip_indexer_csharp`	SCIP indexer for C# projects using scip-dotnet.
`scip_indexer_go`	SCIP indexer for Go projects using scip-go.
`scip_indexer_java`	SCIP indexers for JVM projects using scip-java.
`scip_indexer_php`	SCIP-backed PHP graph routes using scip-php.
`scip_indexer_python`	SCIP indexer for Python projects using scip-python.
`scip_indexer_ruby`	SCIP-backed Ruby graph routes using scip-ruby.
`scip_indexer_rust`	SCIP indexer for Rust projects using rust-analyzer.
`scip_indexer_ts`	SCIP indexer for TypeScript and JavaScript projects using scip-typescript.
`scip_pb2`	Generated protocol buffer code.
`scip_query`	Manifest-bound, graph-free access to the native SCIP FactQueryIndex.
`scip_query_provider`	Experimental lazy LSP provider over an admitted SCIP FactQueryIndex.
`typescript_semantics`	Provider-aware TypeScript facts layered over decoded SCIP text.

Classes:

Name	Description
`SCIPLocation`	Repository-relative location returned by the occurrence index.
`SCIPOccurrence`	One SCIP symbol occurrence with an exact half-open source range.
`SCIPOccurrenceIndex`	Resolve exact source positions to SCIP definitions and references.
`SCIPCSharpGraphDecoder`	Decode scip-dotnet TextFormat output into CodeGraph.
`SCIPJavaGraphDecoder`	Decode scip-java TextFormat output into CodeGraph.
`SCIPPHPGraphDecoder`	Decode scip-php TextFormat output into CodeGraph.
`SCIPPythonGraphDecoder`	Decoder for Python SCIP indices.
`SCIPRubyGraphDecoder`	Decode scip-ruby TextFormat output into CodeGraph.
`SCIPRustGraphDecoder`	Decoder for Rust SCIP indices.
`SCIPTypeScriptGraphDecoder`
`SCIPIndexerBase`	Abstract base class for SCIP indexers.
`SCIPCSharpIndexer`	Run scip-dotnet for the active C# graph backend.
`SCIPJavaIndexer`	Run scip-java for Java and JVM candidate routes.
`SCIPKotlinIndexer`	Candidate scip-java indexer for Kotlin projects.
`SCIPScalaIndexer`	Candidate scip-java indexer for Scala projects.
`PHPHybridIndexer`	Active PHP graph route: SCIP for Composer projects, LSP fallback otherwise.
`SCIPPHPIndexer`	Pure scip-php route used by candidate gates and the active hybrid route.
`SCIPPythonIndexer`	SCIP indexer for Python projects.
`RubyHybridIndexer`	Active Ruby graph route: prepared scip-ruby bundles, LSP fallback otherwise.
`SCIPRubyIndexer`	Pure scip-ruby route used by candidate gates and active hybrid routing.
`SCIPRustIndexer`	SCIP indexer for Rust projects.
`SCIPTypeScriptIndexer`	SCIP indexer for TypeScript and JavaScript projects.

SCIPLocation `dataclass` ¶

SCIPLocation(
    file_path: str,
    start_line: int,
    start_character: int,
    end_line: int,
    end_character: int,
)

Repository-relative location returned by the occurrence index.

SCIPOccurrence `dataclass` ¶

SCIPOccurrence(
    file_path: str,
    start_line: int,
    start_character: int,
    end_line: int,
    end_character: int,
    symbol: str,
    symbol_roles: int = 0,
)

One SCIP symbol occurrence with an exact half-open source range.

SCIPOccurrenceIndex ¶

SCIPOccurrenceIndex(
    occurrences: Iterable[SCIPOccurrence], *, position_encoding: str = "UTF8"
)

Resolve exact source positions to SCIP definitions and references.

Methods:

Name	Description
`from_scip_index`	Use the producer's parsed index without rescanning its text encoding.
`load_stream`	Load an occurrence index from a caller-owned binary stream.

Source code in codenib/scip_interface/lsp_occurrence_index.py

def __init__(
    self,
    occurrences: Iterable[SCIPOccurrence],
    *,
    position_encoding: str = "UTF8",
) -> None:
    self.position_encoding = str(position_encoding or "UTF8")
    self.occurrences = tuple(occurrences)
    by_file: dict[str, list[int]] = {}
    by_symbol: dict[tuple[str, str], list[int]] = {}
    for index, occurrence in enumerate(self.occurrences):
        by_file.setdefault(occurrence.file_path, []).append(index)
        key = self._symbol_key(occurrence.file_path, occurrence.symbol)
        by_symbol.setdefault(key, []).append(index)
    self._by_file = {
        file_path: tuple(
            sorted(
                indexes,
                key=lambda item: self.occurrences[item].span_key,
            )
        )
        for file_path, indexes in by_file.items()
    }
    self._by_symbol = {key: tuple(indexes) for key, indexes in by_symbol.items()}

from_scip_index `classmethod` ¶

from_scip_index(
    index: Index, *, path_filter: Callable[[str], bool] | None = None
) -> SCIPOccurrenceIndex

Use the producer's parsed index without rescanning its text encoding.

Callers that combine this with a graph decoded from a file must bind both consumers to the same serialized index before publication.

Source code in codenib/scip_interface/lsp_occurrence_index.py

@classmethod
def from_scip_index(
    cls,
    index: Index,
    *,
    path_filter: Optional[Callable[[str], bool]] = None,
) -> SCIPOccurrenceIndex:
    """Use the producer's parsed index without rescanning its text encoding.

    Callers that combine this with a graph decoded from a file must bind
    both consumers to the same serialized index before publication.
    """
    from .scip_pb2 import TextEncoding

    try:
        position_encoding = (
            TextEncoding.Name(index.metadata.text_document_encoding)
            if index.metadata.text_document_encoding
            else "UTF8"
        )
    except ValueError:
        value = index.metadata.text_document_encoding
        position_encoding = str(value) if value >= 0 else "UTF8"
    occurrences = []
    for document in index.documents:
        file_path = document.relative_path
        if not file_path or (
            path_filter is not None and not path_filter(file_path)
        ):
            continue
        normalized_path = _normalize_path(file_path)
        for row in document.occurrences:
            values = row.range
            if (
                not row.symbol
                or len(values) not in (3, 4)
                or any(value < 0 for value in values)
            ):
                continue
            start_line, start_character = values[:2]
            end_line, end_character = (
                (start_line, values[2]) if len(values) == 3 else values[2:]
            )
            occurrences.append(
                SCIPOccurrence(
                    file_path=normalized_path,
                    start_line=start_line,
                    start_character=start_character,
                    end_line=end_line,
                    end_character=end_character,
                    symbol=row.symbol,
                    symbol_roles=row.symbol_roles,
                )
            )
    return cls(occurrences, position_encoding=position_encoding)

load_stream `classmethod` ¶

load_stream(
    handle: BinaryIO, *, input_label: str = "<authenticated stream>"
) -> SCIPOccurrenceIndex

Load an occurrence index from a caller-owned binary stream.

Source code in codenib/scip_interface/lsp_occurrence_index.py

@classmethod
def load_stream(
    cls,
    handle: BinaryIO,
    *,
    input_label: str = "<authenticated stream>",
) -> SCIPOccurrenceIndex:
    """Load an occurrence index from a caller-owned binary stream."""

    payload = pickle.load(handle)
    if not isinstance(payload, Mapping):
        raise ValueError(f"invalid LSP occurrence index at {input_label}")
    version = payload.get("schema_version")
    if version != LSP_OCCURRENCE_INDEX_SCHEMA_VERSION:
        raise ValueError(
            f"LSP occurrence index at {input_label} "
            f"has schema_version={version!r}, "
            f"expected {LSP_OCCURRENCE_INDEX_SCHEMA_VERSION}"
        )
    rows = payload.get("occurrences") or ()
    occurrences = tuple(
        row if isinstance(row, SCIPOccurrence) else SCIPOccurrence(**row)
        for row in rows
    )
    by_file = payload.get("by_file")
    by_symbol = payload.get("by_symbol")
    if isinstance(by_file, Mapping) and isinstance(by_symbol, Mapping):
        index = cls.__new__(cls)
        index.position_encoding = str(payload.get("position_encoding") or "UTF8")
        index.occurrences = occurrences
        index._by_file = {str(key): tuple(value) for key, value in by_file.items()}
        index._by_symbol = {
            tuple(key): tuple(value) for key, value in by_symbol.items()
        }
        return index
    return cls(occurrences, position_encoding=payload.get("position_encoding"))

SCIPCSharpGraphDecoder ¶

SCIPCSharpGraphDecoder(index_file_path, project_root=None)

Bases: SCIPJavaGraphDecoder

Decode scip-dotnet TextFormat output into CodeGraph.

Source code in codenib/scip_interface/scip_decode_java.py

def __init__(self, index_file_path, project_root=None):
    self.index_file_path = index_file_path
    self.project_root = Path(project_root) if project_root else None
    self.code_graph = CodeGraph(project_root)
    self.indexed_directories: set[str] = set()
    self.symbols: dict[str, _SymbolInfo] = {}
    self.definition_lines: dict[str, list[_DefinitionInfo]] = {}
    self.logger = get_logger(__name__)
    register_scip_logger(__name__)

SCIPJavaGraphDecoder ¶

SCIPJavaGraphDecoder(index_file_path, project_root=None)

Decode scip-java TextFormat output into CodeGraph.

Source code in codenib/scip_interface/scip_decode_java.py

def __init__(self, index_file_path, project_root=None):
    self.index_file_path = index_file_path
    self.project_root = Path(project_root) if project_root else None
    self.code_graph = CodeGraph(project_root)
    self.indexed_directories: set[str] = set()
    self.symbols: dict[str, _SymbolInfo] = {}
    self.definition_lines: dict[str, list[_DefinitionInfo]] = {}
    self.logger = get_logger(__name__)
    register_scip_logger(__name__)

SCIPPHPGraphDecoder ¶

SCIPPHPGraphDecoder(index_file_path, project_root=None)

Bases: SCIPJavaGraphDecoder

Decode scip-php TextFormat output into CodeGraph.

Source code in codenib/scip_interface/scip_decode_java.py

def __init__(self, index_file_path, project_root=None):
    self.index_file_path = index_file_path
    self.project_root = Path(project_root) if project_root else None
    self.code_graph = CodeGraph(project_root)
    self.indexed_directories: set[str] = set()
    self.symbols: dict[str, _SymbolInfo] = {}
    self.definition_lines: dict[str, list[_DefinitionInfo]] = {}
    self.logger = get_logger(__name__)
    register_scip_logger(__name__)

SCIPPythonGraphDecoder ¶

SCIPPythonGraphDecoder(index_file_path, project_root=None)

Decoder for Python SCIP indices.

Parses decoded SCIP index files and builds a CodeGraph representing: - Classes - Methods and fields within classes - Module-level functions - References between symbols

Parameters:

Name	Type	Description	Default
`index_file_path`		Path to the decoded SCIP index file	required
`project_root`		Root directory of the Python project	`None`

Methods:

Name	Description
`decode`	Decode the SCIP index and build the CodeGraph.
`save_graph`	Save the code graph to a file.

Source code in codenib/scip_interface/scip_decode_python.py

def __init__(self, index_file_path, project_root=None):
    """
    Initialize the Python SCIP decoder.

    Args:
        index_file_path: Path to the decoded SCIP index file
        project_root: Root directory of the Python project
    """
    self.index_file_path = index_file_path
    self.project_root = project_root
    self.code_graph = CodeGraph(project_root)
    self.indexed_directories = set()
    self.logger = get_logger(__name__)
    register_scip_logger(__name__)

decode ¶

decode()

Decode the SCIP index and build the CodeGraph.

Returns:

Name	Type	Description
`CodeGraph`		The constructed code graph

Source code in codenib/scip_interface/scip_decode_python.py

def decode(self):
    """
    Decode the SCIP index and build the CodeGraph.

    Returns:
        CodeGraph: The constructed code graph
    """
    self.logger.info(f"Starting SCIP Python decode from {self.index_file_path}")
    try:
        with open(self.index_file_path, "r") as f:
            content = f.read()
    except Exception as e:
        self.logger.error(f"Error reading SCIP index file: {e}")
        raise

    # Parse documents
    document_blocks = re.findall(
        r"documents\s*{(.*?)(?=documents\s*{|$)", content, re.DOTALL
    )

    # Add the root node to the graph
    self.code_graph.add_root_node(ROOT_NODE)

    # Process all documents. Batch edge insertion: per-edge add_edges is
    # O(E^2) (igraph reindexes each call); buffering flushes once as O(E).
    with self.code_graph.batch_edges():
        for document in document_blocks:
            self._process_document(document)

    self._fix_unified_names()
    return self.code_graph

save_graph ¶

save_graph(output_path)

Save the code graph to a file.

Parameters:

Name	Type	Description	Default
`output_path`		Path where the graph should be saved	required

Source code in codenib/scip_interface/scip_decode_python.py

def save_graph(self, output_path):
    """
    Save the code graph to a file.

    Args:
        output_path: Path where the graph should be saved
    """
    self.code_graph.save_graph(output_path)

SCIPRubyGraphDecoder ¶

SCIPRubyGraphDecoder(index_file_path, project_root=None)

Bases: SCIPJavaGraphDecoder

Decode scip-ruby TextFormat output into CodeGraph.

Source code in codenib/scip_interface/scip_decode_ruby.py

def __init__(self, index_file_path, project_root=None):
    super().__init__(index_file_path, project_root=project_root)
    self._file_definition_keys: dict[tuple[str, str], str] = {}
    self._display_names_by_graph_key: dict[str, str] = {}
    self._source_lines_by_file: dict[str, tuple[str, ...] | None] = {}

SCIPRustGraphDecoder ¶

SCIPRustGraphDecoder(index_file_path, project_root=None)

Decoder for Rust SCIP indices.

Parses decoded SCIP index files and builds a CodeGraph representing: - Structs as classes - Traits as classes - Impl blocks and their methods - Free functions - References between symbols

Parameters:

Name	Type	Description	Default
`index_file_path`		Path to the decoded SCIP index file	required
`project_root`		Root directory of the Rust project	`None`

Methods:

Name	Description
`decode`	Decode the SCIP index and build the CodeGraph.
`save_graph`	Save the code graph to a file.

Source code in codenib/scip_interface/scip_decode_rust.py

def __init__(self, index_file_path, project_root=None):
    """
    Initialize the Rust SCIP decoder.

    Args:
        index_file_path: Path to the decoded SCIP index file
        project_root: Root directory of the Rust project
    """
    self.index_file_path = index_file_path
    self.project_root = Path(project_root) if project_root else None
    self.code_graph = CodeGraph(project_root)
    self.indexed_directories = set()
    self.logger = get_logger(__name__)
    # Register this module for SCIP debug logging
    register_scip_logger(__name__)

    # Internal workspace crate names (loaded lazily on first document)
    self.internal_crates = None

decode ¶

decode()

Decode the SCIP index and build the CodeGraph.

Returns:

Name	Type	Description
`CodeGraph`		The constructed code graph

Source code in codenib/scip_interface/scip_decode_rust.py

def decode(self):
    """
    Decode the SCIP index and build the CodeGraph.

    Returns:
        CodeGraph: The constructed code graph
    """
    self.logger.info(f"Starting SCIP Rust decode from {self.index_file_path}")
    try:
        with open(self.index_file_path, "r") as f:
            content = f.read()
    except Exception as e:
        self.logger.error(f"Error reading SCIP index file: {e}")
        raise

    # Parse documents
    document_blocks = re.findall(
        r"documents\s*{(.*?)(?=documents\s*{|$)", content, re.DOTALL
    )

    # Add the root node to the graph
    self.code_graph.add_root_node(ROOT_NODE)

    # Process all documents. Batch edge insertion: per-edge add_edges is
    # O(E^2) (igraph reindexes each call); buffering flushes once as O(E).
    with self.code_graph.batch_edges():
        for document in document_blocks:
            self._process_document(document)

    return self.code_graph

save_graph ¶

save_graph(output_path)

Save the code graph to a file.

Parameters:

Name	Type	Description	Default
`output_path`		Path where the graph should be saved	required

Source code in codenib/scip_interface/scip_decode_rust.py

def save_graph(self, output_path):
    """
    Save the code graph to a file.

    Args:
        output_path: Path where the graph should be saved
    """
    self.code_graph.save_graph(output_path)

SCIPTypeScriptGraphDecoder ¶

SCIPTypeScriptGraphDecoder(index_file_path, project_root=None)

Source code in codenib/scip_interface/scip_decode_ts.py

def __init__(self, index_file_path, project_root=None):
    self.index_file_path = index_file_path
    self.project_root = project_root
    self.code_graph = CodeGraph(project_root)
    self.indexed_directories = set()
    self.logger = get_logger(__name__)
    # Register this module for SCIP debug logging
    register_scip_logger(__name__)

    # Track actual file paths from documents to fix index file handling
    self.document_file_paths = set()

    # Track project's own SCIP package names to filter external refs
    self._project_packages = set()
    self._symbol_kinds = {}

SCIPIndexerBase ¶

SCIPIndexerBase(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    language: str = "unknown",
    decoder_backend: str | None = None,
)

Bases: ABC

Abstract base class for SCIP indexers.

This class provides common functionality for all SCIP indexers, while allowing language-specific implementations to customize the indexing process.

Parameters:

Name	Type	Description	Default
`project_root`	`str \| Path`	Root directory of the project to index	required
`output_dir`	`str \| Path \| None`	Directory to store output files	`None`
`exclude_patterns`	`list \| None`	List of patterns to exclude from indexing	`None`
`profiler`	`Profiler \| None`	Profiler instance for performance tracking	`None`
`language`	`str`	Language being indexed (for logging)	`'unknown'`
`decoder_backend`	`str \| None`	Which decoder to use when `process_index` runs. `"serial"` (default) uses the pure-Python per-language decoder; `"core"` uses the C++ pybind decoder from `core/`.	`None`

Methods:

Name	Description
`generate_index`	Generate SCIP index for the project.
`decode_index`	Decode the SCIP index with the packaged protobuf descriptor.
`process_index`	Process the decoded SCIP index into a more usable format.
`run_pipeline`	Run the complete SCIP indexing pipeline: generate, decode, and process.
`clear_cache`	Clear cache files at different levels.

Source code in codenib/scip_interface/scip_indexer_base.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    language: str = "unknown",
    decoder_backend: Optional[str] = None,
):
    """
    Initialize the SCIP indexer.

    Args:
        project_root: Root directory of the project to index
        output_dir: Directory to store output files
        exclude_patterns: List of patterns to exclude from indexing
        profiler: Profiler instance for performance tracking
        language: Language being indexed (for logging)
        decoder_backend: Which decoder to use when ``process_index`` runs.
            ``"serial"`` (default) uses the pure-Python per-language decoder;
            ``"core"`` uses the C++ pybind decoder from ``core/``.
    """
    self.project_root = Path(project_root).absolute()
    self.language = language
    self.decoder_backend = self._resolve_backend(decoder_backend)

    if output_dir:
        self.output_dir = Path(output_dir).absolute()
    else:
        self.output_dir = temp_state_dir() / self.project_root.name

    # Create output directory if it doesn't exist
    os.makedirs(self.output_dir, exist_ok=True)

    # Set paths for index files in the output directory
    self.index_file = self.output_dir / "index.scip"
    self.decoded_file = self.output_dir / "index.decoded"
    self.graph_file = self.output_dir / "graph.pkl"
    self.lsp_index_file = self.output_dir / "lsp_index.pkl"
    self.exclude_patterns = exclude_patterns if exclude_patterns else []
    self._target_dir: str | None = None
    self.decoded_input_receipt: dict[str, object] | None = None
    self.graph_output_receipt: dict[str, object] | None = None
    self._capture_artifact_receipts = False
    self.profiler = profiler or Profiler(f"scip_{language}_indexer")

    # Path to the scip.proto file (shared across all indexers)
    self.module_dir = Path(__file__).parent
    self.proto_file = self.module_dir / "scip.proto"

generate_index ¶

generate_index(**kwargs) -> bool

Generate SCIP index for the project.

Parameters:

Name	Type	Description	Default
`**kwargs`		Language-specific options	`{}`

Returns:

Name	Type	Description
`bool`	`bool`	True if index generation was successful, False otherwise

Source code in codenib/scip_interface/scip_indexer_base.py

def generate_index(self, **kwargs) -> bool:
    """
    Generate SCIP index for the project.

    Args:
        **kwargs: Language-specific options

    Returns:
        bool: True if index generation was successful, False otherwise
    """
    if not self._check_indexer_available():
        return False

    # Build the command
    cmd = self._build_index_command(**kwargs)

    logger.debug(f"Running command: {' '.join(cmd)}")

    # Run the command
    with self.profiler.section("generate_index") as section:
        try:
            # For Rust projects, override the repository toolchain so SCIP
            # generation uses CodeNib's selected rust-analyzer version.
            env = None
            if self.language == "rust":
                import os

                from .rust_analyzer import rust_toolchain

                env = os.environ.copy()
                env["RUSTUP_TOOLCHAIN"] = rust_toolchain()

            subprocess.run(
                cmd,
                check=True,
                cwd=self.project_root,
                capture_output=True,
                text=True,
                env=env,
            )
            success = True
        except subprocess.CalledProcessError as e:
            logger.error(f"Error generating SCIP index: {e}")
            if e.stdout:
                logger.error(f"stdout: {e.stdout}")
            if e.stderr:
                logger.error(f"stderr: {e.stderr}")
            success = False

    duration = section.duration

    if success:
        logger.info(f"Successfully generated SCIP index at {self.index_file}")
        logger.info(f"⏱️  Index generation took: {duration:.2f} seconds")
        return True
    else:
        logger.error(f"❌ Index generation failed after {duration:.2f} seconds")
        return False

decode_index ¶

decode_index() -> bool

Decode the SCIP index with the packaged protobuf descriptor.

Returns:

Name	Type	Description
`bool`	`bool`	True if decoding was successful, False otherwise

Source code in codenib/scip_interface/scip_indexer_base.py

def decode_index(self) -> bool:
    """
    Decode the SCIP index with the packaged protobuf descriptor.

    Returns:
        bool: True if decoding was successful, False otherwise
    """
    if not self.index_file.exists():
        logger.error(f"Index file not found at {self.index_file}")
        return False

    try:
        from google.protobuf.message import DecodeError
        from google.protobuf.text_format import MessageToString

        from .scip_pb2 import Index
    except ImportError:
        logger.error("SCIP decoding requires protobuf. Install `codenib[graph]`.")
        return False

    try:
        with self.profiler.section("decode_index") as section:
            payload = self.index_file.read_bytes()
            if not payload:
                raise DecodeError("SCIP index is empty")
            index = Index()
            index.ParseFromString(payload)
            self.decoded_file.write_text(
                MessageToString(index, as_utf8=True),
                encoding="utf-8",
            )
        duration = section.duration

        logger.info(f"Successfully decoded SCIP index to {self.decoded_file}")
        logger.info(f"⏱️  Index decoding took: {duration:.2f} seconds")
        return True
    except (OSError, DecodeError) as e:
        logger.error(f"Error decoding SCIP index: {e}")
        return False

process_index ¶

process_index(output_file: str | None = None) -> CodeGraph | None

Process the decoded SCIP index into a more usable format.

Parameters:

Name	Type	Description	Default
`output_file`	`str \| None`	Path to write the processed data to	`None`

Returns:

Name	Type	Description
`CodeGraph`	`CodeGraph \| None`	Processed graph object

Source code in codenib/scip_interface/scip_indexer_base.py

def process_index(
    self, output_file: Optional[str] = None
) -> Union[CodeGraph, None]:
    """
    Process the decoded SCIP index into a more usable format.

    Args:
        output_file: Path to write the processed data to

    Returns:
        CodeGraph: Processed graph object
    """
    return self._process_decoded_index(output_file)

run_pipeline ¶

run_pipeline(
    output_file: str | None = None,
    skip_level: str | None = None,
    *,
    reset_profiler: bool = True,
    report_profile: bool = True,
    **kwargs
) -> CodeGraph | None

Run the complete SCIP indexing pipeline: generate, decode, and process.

Parameters:

Name	Type	Description	Default
`output_file`	`str \| None`	Path to write the processed data to (if None, uses self.graph_file)	`None`
`skip_level`	`str \| None`	Cache/skip level - 'graph', 'decode', 'raw', or None - 'graph': Check if graph.pkl exists, load and return it if found - 'decode': Check if index.decoded exists, skip to processing if found - 'raw': Check if index.scip exists, skip to decoding if found - None: Run full pipeline from scratch (default)	`None`
`reset_profiler`	`bool`	Clear profiler stats before running the pipeline	`True`
`report_profile`	`bool`	Emit profiler summary automatically after the run	`True`
`**kwargs`		Language-specific options passed to generate_index	`{}`

Returns:

Name	Type	Description
`CodeGraph`	`CodeGraph \| None`	Processed graph object

Source code in codenib/scip_interface/scip_indexer_base.py

def run_pipeline(
    self,
    output_file: Optional[str] = None,
    skip_level: Optional[str] = None,
    *,
    reset_profiler: bool = True,
    report_profile: bool = True,
    **kwargs,
) -> Union[CodeGraph, None]:
    """
    Run the complete SCIP indexing pipeline: generate, decode, and process.

    Args:
        output_file: Path to write the processed data to (if None, uses self.graph_file)
        skip_level: Cache/skip level - 'graph', 'decode', 'raw', or None
            - 'graph': Check if graph.pkl exists, load and return it if found
            - 'decode': Check if index.decoded exists, skip to processing if found
            - 'raw': Check if index.scip exists, skip to decoding if found
            - None: Run full pipeline from scratch (default)
        reset_profiler: Clear profiler stats before running the pipeline
        report_profile: Emit profiler summary automatically after the run
        **kwargs: Language-specific options passed to generate_index

    Returns:
        CodeGraph: Processed graph object
    """
    # Use default graph file if output_file not specified
    self.decoded_input_receipt = None
    self.graph_output_receipt = None
    capture_artifact_receipts = kwargs.pop("capture_artifact_receipts", False)
    if type(capture_artifact_receipts) is not bool:
        raise ValueError("capture_artifact_receipts must be a boolean")
    self._capture_artifact_receipts = capture_artifact_receipts
    if output_file is None:
        output_file = str(self.graph_file)
    self._target_dir = normalize_scip_path(kwargs.pop("target_dir", None))

    # Check graph cache if skip_level is 'graph'
    if skip_level == "graph" and self.graph_file.exists():
        logger.info(f"Loading cached graph from {self.graph_file}")
        try:
            from ..graph.code_graph import CodeGraph

            graph = CodeGraph.load_graph(str(self.graph_file))
            if self.lsp_index_file.is_file():
                from .lsp_occurrence_index import SCIPOccurrenceIndex

                graph.lsp_occurrence_index = SCIPOccurrenceIndex.load(
                    self.lsp_index_file
                )
            elif self.decoded_file.is_file():
                from .lsp_occurrence_index import SCIPOccurrenceIndex

                graph.lsp_occurrence_index = SCIPOccurrenceIndex.from_decoded_file(
                    self.decoded_file,
                    path_filter=(
                        self._path_allowed
                        if self._target_dir or self.exclude_patterns
                        else None
                    ),
                )
                graph.lsp_occurrence_index.save(self.lsp_index_file)
            if self._target_dir or self.exclude_patterns:
                graph = self._filter_project_graph(graph)
                graph.build_range_indexes()
                if output_file:
                    graph.save_graph(output_file)
                    if hasattr(graph, "lsp_occurrence_index"):
                        graph.lsp_occurrence_index.save(
                            Path(output_file).with_name("lsp_index.pkl")
                        )
            logger.info(
                "✅ Successfully loaded cached graph "
                f"({len(graph.graph.vs)} nodes, "
                f"{len(graph.graph.es)} edges)"
            )
            return graph
        except Exception as e:
            logger.warning(
                f"Failed to load cached graph: {e}. Proceeding with pipeline..."
            )

    # Determine what needs to be generated based on what exists
    if skip_level in ("graph", "decode") and self.decoded_file.exists():
        logger.info(
            f"Found existing decoded file at "
            f"{self.decoded_file}, skipping generation and decode"
        )
        should_generate_index = False
        should_decode_index = False
    elif skip_level in ("graph", "decode", "raw") and self.index_file.exists():
        logger.info(
            f"Found existing raw index at {self.index_file}, skipping generation"
        )
        should_generate_index = False
        should_decode_index = True
    else:
        should_generate_index = True
        should_decode_index = True

    if reset_profiler:
        self.profiler.reset()

    try:
        # Generate the index if needed
        if should_generate_index:
            logger.info("Generating SCIP index")
            if not self.generate_index(**kwargs):
                # The indexer reported failure but the index
                # file may still have been written
                # (e.g. scip-typescript crashes during cleanup after emitting index.scip).
                # Continue if the file exists.
                if not self.index_file.exists():
                    return None
                logger.warning(
                    "Index generation returned failure but %s exists; continuing.",
                    self.index_file,
                )

        # Decode the index if needed
        if should_decode_index:
            if not self.index_file.exists():
                logger.error(
                    f"Index file not found at {self.index_file}, cannot decode"
                )
                return None
            logger.info("Decoding SCIP index")
            if not self.decode_index():
                return None

        # Process the index and save graph
        graph = self.process_index(output_file)

        if graph:
            logger.info(
                "✅ Graph created successfully "
                f"({len(graph.graph.vs)} nodes, "
                f"{len(graph.graph.es)} edges)"
            )

        return graph
    finally:
        self._capture_artifact_receipts = False
        if report_profile:
            self.profiler.report(reset=reset_profiler)

clear_cache ¶

clear_cache(level: str = 'all') -> bool

Clear cache files at different levels.

Parameters:

Name	Type	Description	Default
`level`	`str`	Preserve cache up to this pipeline stage, remove above. Pipeline: raw (index.scip) → decode (index.decoded) → graph (graph.pkl) - 'graph': keep everything (raw + decoded + graph) - 'decode': keep raw + decoded, remove graph - 'raw': keep raw, remove decoded + graph - 'all': remove all cache files (default)	`'all'`

Returns:

Name	Type	Description
`bool`	`bool`	True if cache clearing was successful, False otherwise

Source code in codenib/scip_interface/scip_indexer_base.py

def clear_cache(self, level: str = "all") -> bool:
    """
    Clear cache files at different levels.

    Args:
        level: Preserve cache up to this pipeline stage, remove above.
            Pipeline: raw (index.scip) → decode (index.decoded) → graph (graph.pkl)
            - 'graph': keep everything (raw + decoded + graph)
            - 'decode': keep raw + decoded, remove graph
            - 'raw': keep raw, remove decoded + graph
            - 'all': remove all cache files (default)

    Returns:
        bool: True if cache clearing was successful, False otherwise
    """
    try:
        files_to_remove = []

        if level == "graph":
            files_to_remove = []
            logger.info("Clearing cache: keeping up to graph (nothing to remove)")
        elif level == "decode":
            files_to_remove = [self.graph_file, self.lsp_index_file]
            logger.info("Clearing cache: keeping up to decode, removing graph")
        elif level == "raw":
            files_to_remove = [
                self.decoded_file,
                self.graph_file,
                self.lsp_index_file,
            ]
            logger.info("Clearing cache: keeping raw, removing decoded + graph")
        elif level == "all":
            files_to_remove = [
                self.index_file,
                self.decoded_file,
                self.graph_file,
                self.lsp_index_file,
            ]
            logger.info("Clearing all cache files")
        else:
            logger.error(
                f"Invalid cache level: {level}. Must be 'graph', 'decode', 'raw', or 'all'"
            )
            return False

        # Remove the specified files
        for file_path in files_to_remove:
            if file_path.exists():
                file_path.unlink()
                logger.info(f"Removed {file_path}")
            else:
                logger.debug(f"File does not exist, skipping: {file_path}")

        logger.info("✅ Cache cleared successfully")
        return True

    except Exception as e:
        logger.error(f"Error clearing cache: {e}")
        return False

SCIPCSharpIndexer ¶

SCIPCSharpIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Bases: SCIPIndexerBase

Run scip-dotnet for the active C# graph backend.

Source code in codenib/scip_interface/scip_indexer_csharp.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        language="csharp",
        decoder_backend=decoder_backend,
    )

SCIPJavaIndexer ¶

SCIPJavaIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
    scip_language: str = "java",
)

Bases: SCIPIndexerBase

Run scip-java for Java and JVM candidate routes.

Source code in codenib/scip_interface/scip_indexer_java.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
    scip_language: str = "java",
):
    self.scip_language = scip_language
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        language=scip_language,
        decoder_backend=decoder_backend,
    )

SCIPKotlinIndexer ¶

SCIPKotlinIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Bases: SCIPJavaIndexer

Candidate scip-java indexer for Kotlin projects.

Source code in codenib/scip_interface/scip_indexer_java.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        decoder_backend=decoder_backend,
        scip_language="kotlin",
    )

SCIPScalaIndexer ¶

SCIPScalaIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Bases: SCIPJavaIndexer

Candidate scip-java indexer for Scala projects.

Source code in codenib/scip_interface/scip_indexer_java.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        decoder_backend=decoder_backend,
        scip_language="scala",
    )

PHPHybridIndexer ¶

PHPHybridIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Active PHP graph route: SCIP for Composer projects, LSP fallback otherwise.

Source code in codenib/scip_interface/scip_indexer_php.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    self.project_root = Path(project_root).absolute()
    self.output_dir = (
        Path(output_dir).absolute()
        if output_dir
        else temp_state_dir() / self.project_root.name
    )
    self.exclude_patterns = exclude_patterns or []
    self.profiler = profiler
    self.decoder_backend = decoder_backend
    self._delegate = self._preferred_delegate()
    self._sync_delegate_attrs()

SCIPPHPIndexer ¶

SCIPPHPIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Bases: SCIPIndexerBase

Pure scip-php route used by candidate gates and the active hybrid route.

Source code in codenib/scip_interface/scip_indexer_php.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    self._index_via_docker = False
    self._index_root: Path | None = None
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        language="php",
        decoder_backend=decoder_backend,
    )

SCIPPythonIndexer ¶

SCIPPythonIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Bases: SCIPIndexerBase

SCIP indexer for Python projects.

Uses a scip-python executable already available on PATH when possible. The historical managed Conda environment remains a fallback for development and CI environments that do not expose the tool directly.

Parameters:

Name	Type	Description	Default
`project_root`	`str \| Path`	Root directory of the Python project	required
`output_dir`	`str \| Path \| None`	Directory to store output files	`None`
`exclude_patterns`	`list \| None`	List of patterns to exclude from indexing	`None`
`profiler`	`Profiler \| None`	Profiler instance for performance tracking	`None`
`decoder_backend`	`str \| None`	`"serial"` (default) or `"core"`.	`None`

Methods:

Name	Description
`generate_index`	Generate SCIP index for the Python project.
`process_index`	Recover identities before serial/core share the decoded input.
`run_pipeline`	Run Python pipeline while ignoring non-Python kwargs.

Source code in codenib/scip_interface/scip_indexer_python.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    """
    Initialize the Python SCIP indexer.

    Args:
        project_root: Root directory of the Python project
        output_dir: Directory to store output files
        exclude_patterns: List of patterns to exclude from indexing
        profiler: Profiler instance for performance tracking
        decoder_backend: ``"serial"`` (default) or ``"core"``.
    """
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        language="python",
        decoder_backend=decoder_backend,
    )

    # Conda environment configuration
    self.conda_env_name = "scip-env"
    self.env_file = self.module_dir / "scip-environment.yml"
    self._direct_indexer_path: Optional[str] = None
    self.index_generation_report: Optional[dict] = None

generate_index ¶

generate_index(
    cwd: str | None = None,
    project_name: str | None = None,
    target_dir: str | None = None,
    allow_partial_index: bool = False,
    **kwargs
) -> bool

Generate SCIP index for the Python project.

Uses the resolved PATH executable, with Conda as a compatibility fallback.

Parameters:

Name	Type	Description	Default
`cwd`	`str \| None`	Working directory (defaults to project_root)	`None`
`project_name`	`str \| None`	Project name to use in the index	`None`
`target_dir`	`str \| None`	Optional subdirectory to target for indexing	`None`
`allow_partial_index`	`bool`	Preserve a parseable compiler prefix for downstream source-coverage repair.	`False`
`**kwargs`		Additional arguments (ignored)	`{}`

Returns:

Name	Type	Description
`bool`	`bool`	True if index generation was successful, False otherwise

Source code in codenib/scip_interface/scip_indexer_python.py

def generate_index(
    self,
    cwd: Optional[str] = None,
    project_name: Optional[str] = None,
    target_dir: Optional[str] = None,
    allow_partial_index: bool = False,
    **kwargs,
) -> bool:
    """
    Generate SCIP index for the Python project.

    Uses the resolved PATH executable, with Conda as a compatibility
    fallback.

    Args:
        cwd: Working directory (defaults to project_root)
        project_name: Project name to use in the index
        target_dir: Optional subdirectory to target for indexing
        allow_partial_index: Preserve a parseable compiler prefix for
            downstream source-coverage repair.
        **kwargs: Additional arguments (ignored)

    Returns:
        bool: True if index generation was successful, False otherwise
    """
    self.index_generation_report = None
    if self.index_file.exists():
        self.index_file.unlink()
        logger.info("Removed pre-existing SCIP index before generation")
    pending = self._pending_index_file
    pending.unlink(missing_ok=True)
    if not self._check_indexer_available():
        self.index_generation_report = {
            "backend": "scip-python",
            "status": "unavailable",
            "complete": False,
            "partial": False,
            "document_count": 0,
        }
        return False

    cmd = self._build_index_command(
        cwd=cwd or str(self.project_root),
        project_name=project_name,
        target_dir=target_dir,
        output_file=pending,
    )

    logger.debug(f"Running command: {' '.join(cmd)}")

    with self.profiler.section("generate_index") as section:
        with self._temporary_exclude_config():
            if self._direct_indexer_path:
                success = self._run_direct(cmd, self.project_root)
            else:
                success = self._run_in_conda_env(cmd, self.project_root)
    duration = section.duration

    if success:
        if pending.exists():
            os.replace(pending, self.index_file)
        document_count = self._usable_index_document_count()
        self.index_generation_report = {
            "backend": "scip-python",
            "status": "complete",
            "complete": True,
            "partial": False,
            "document_count": document_count,
        }
        logger.info(f"Successfully generated SCIP index at {self.index_file}")
        logger.info(f"Index generation took: {duration:.2f} seconds")
        return True
    else:
        logger.error(f"Index generation failed after {duration:.2f} seconds")
        document_count = self._usable_index_document_count(pending)
        if allow_partial_index and document_count > 0:
            os.replace(pending, self.index_file)
            self.index_generation_report = {
                "backend": "scip-python",
                "status": "partial",
                "complete": False,
                "partial": True,
                "document_count": document_count,
            }
            logger.warning(
                "Preserving parseable partial SCIP index with %d documents",
                document_count,
            )
            return False
        # Never let the generic pipeline mistake an unvalidated partial file
        # for a complete graph unless the caller explicitly opted into repair.
        if pending.exists():
            pending.unlink()
            logger.info("Removed partial index file")
        self.index_generation_report = {
            "backend": "scip-python",
            "status": "failed",
            "complete": False,
            "partial": False,
            "document_count": document_count,
        }
        return False

process_index ¶

process_index(output_file: str | None = None)

Recover identities before serial/core share the decoded input.

The compiler owns publication of the output directory. Do not replace a decoded input that changed during recovery; the derived-file publish point is the final atomic replace, before the shared consumer seals it.

Source code in codenib/scip_interface/scip_indexer_python.py

def process_index(self, output_file: Optional[str] = None):
    """Recover identities before serial/core share the decoded input.

    The compiler owns publication of the output directory. Do not replace
    a decoded input that changed during recovery; the derived-file publish
    point is the final atomic replace, before the shared consumer seals it.
    """

    from google.protobuf.text_format import MessageToString

    from .lsp_occurrence_index import SCIPOccurrenceIndex
    from .python_identity import normalize_python_index

    self.decoded_input_receipt = None
    self.graph_output_receipt = None
    temporary = None
    try:
        from ..compiler.artifact_fingerprints import regular_file_fingerprint

        before = regular_file_fingerprint(self.decoded_file)
        payload = self.decoded_file.read_bytes()
        if before != {
            "size": len(payload),
            "sha256": hashlib.sha256(payload).hexdigest(),
        }:
            raise RuntimeError(
                "decoded SCIP input changed before identity recovery"
            )
        with self.profiler.section("process_index.read_python_identity_index"):
            index = self._read_identity_index(payload)
        del payload
        with self.profiler.section("process_index.python_identity"):
            report = normalize_python_index(
                index,
                self.project_root,
                path_allowed=self._path_allowed,
            )
        encoded = MessageToString(index, as_utf8=True).encode("utf-8")
        normalized_fingerprint = {
            "size": len(encoded),
            "sha256": hashlib.sha256(encoded).hexdigest(),
        }
        with tempfile.NamedTemporaryFile(
            mode="wb",
            dir=self.decoded_file.parent,
            prefix=".python-identity-",
            delete=False,
        ) as handle:
            temporary = Path(handle.name)
            handle.write(encoded)
        del encoded
        with self.profiler.section("process_index.prepare_python_occurrences"):
            occurrence_index = SCIPOccurrenceIndex.from_scip_index(
                index,
                path_filter=(
                    self._path_allowed
                    if self._target_dir or self.exclude_patterns
                    else None
                ),
            )
        del index
        if regular_file_fingerprint(self.decoded_file) != before:
            raise RuntimeError(
                "decoded SCIP input changed during identity recovery"
            )
        os.replace(temporary, self.decoded_file)
        if self.index_generation_report is not None:
            self.index_generation_report["identity_recovery"] = {
                "status": "complete",
                **report,
            }
    except Exception as exc:
        logger.error("Python SCIP identity recovery failed: %s", exc)
        if self.index_generation_report is not None:
            self.index_generation_report["identity_recovery"] = {
                "status": "failed",
                "error": str(exc),
            }
        return None
    finally:
        if temporary is not None:
            temporary.unlink(missing_ok=True)
    return self._process_decoded_index(
        output_file,
        occurrence_index=occurrence_index,
        decoded_fingerprint=normalized_fingerprint,
    )

run_pipeline ¶

run_pipeline(
    output_file: str | None = None,
    skip_level: str | None = None,
    *,
    reset_profiler: bool = True,
    report_profile: bool = True,
    **kwargs
)

Run Python pipeline while ignoring non-Python kwargs.

Source code in codenib/scip_interface/scip_indexer_python.py

def run_pipeline(
    self,
    output_file: Optional[str] = None,
    skip_level: Optional[str] = None,
    *,
    reset_profiler: bool = True,
    report_profile: bool = True,
    **kwargs,
):
    """
    Run Python pipeline while ignoring non-Python kwargs.
    """
    # Pop kwargs from other languages
    kwargs.pop("config_path", None)
    kwargs.pop("exclude_vendored_libraries", None)
    kwargs.pop("infer_tsconfig", None)
    kwargs.pop("yarn_workspaces", None)
    kwargs.pop("pnpm_workspaces", None)
    kwargs.pop("npm_workspaces", None)
    kwargs.pop("compdb_path", None)
    kwargs.pop("show_compiler_diagnostics", None)

    if skip_level is not None:
        self._discard_index_without_documents()
    return super().run_pipeline(
        output_file=output_file,
        skip_level=skip_level,
        reset_profiler=reset_profiler,
        report_profile=report_profile,
        **kwargs,
    )

RubyHybridIndexer ¶

RubyHybridIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Active Ruby graph route: prepared scip-ruby bundles, LSP fallback otherwise.

Source code in codenib/scip_interface/scip_indexer_ruby.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    self.project_root = Path(project_root).absolute()
    self.output_dir = (
        Path(output_dir).absolute()
        if output_dir
        else temp_state_dir() / self.project_root.name
    )
    self.exclude_patterns = exclude_patterns or []
    self.profiler = profiler
    self.decoder_backend = decoder_backend
    self._delegate = self._preferred_delegate()
    self._sync_delegate_attrs()

SCIPRubyIndexer ¶

SCIPRubyIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Bases: SCIPIndexerBase

Pure scip-ruby route used by candidate gates and active hybrid routing.

Source code in codenib/scip_interface/scip_indexer_ruby.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        language="ruby",
        decoder_backend=decoder_backend,
    )

SCIPRustIndexer ¶

SCIPRustIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Bases: SCIPIndexerBase

SCIP indexer for Rust projects.

Uses the rust-analyzer tool to generate SCIP indices for Rust codebases. rust-analyzer scip generates a SCIP index from Rust source code.

Parameters:

Name	Type	Description	Default
`project_root`	`str \| Path`	Root directory of the Rust project (must contain Cargo.toml)	required
`output_dir`	`str \| Path \| None`	Directory to store output files (defaults to /tmp/project_name)	`None`
`exclude_patterns`	`list \| None`	List of patterns to exclude from indexing	`None`
`profiler`	`Profiler \| None`	Profiler instance for performance tracking	`None`
`decoder_backend`	`str \| None`	`"serial"` (default) or `"core"`.	`None`

Methods:

Name	Description
`generate_index`	Generate SCIP index for the Rust project.
`run_pipeline`	Run Rust pipeline while ignoring Python-specific kwargs.

Source code in codenib/scip_interface/scip_indexer_rust.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    """
    Initialize the Rust SCIP indexer.

    Args:
        project_root: Root directory of the Rust project (must contain Cargo.toml)
        output_dir: Directory to store output files (defaults to /tmp/project_name)
        exclude_patterns: List of patterns to exclude from indexing
        profiler: Profiler instance for performance tracking
        decoder_backend: ``"serial"`` (default) or ``"core"``.
    """
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        language="rust",
        decoder_backend=decoder_backend,
    )
    self.index_generation_report: Optional[dict] = None

generate_index ¶

generate_index(
    config_path: str | None = None, exclude_vendored_libraries: bool = False
) -> bool

Generate SCIP index for the Rust project.

Parameters:

Name	Type	Description	Default
`config_path`	`str \| None`	Path to a JSON configuration file for customizing cargo behavior	`None`
`exclude_vendored_libraries`	`bool`	Exclude code from vendored libraries from the index	`False`

Returns:

Name	Type	Description
`bool`	`bool`	True if index generation was successful, False otherwise

Source code in codenib/scip_interface/scip_indexer_rust.py

def generate_index(
    self,
    config_path: Optional[str] = None,
    exclude_vendored_libraries: bool = False,
) -> bool:
    """
    Generate SCIP index for the Rust project.

    Args:
        config_path: Path to a JSON configuration file for customizing cargo behavior
        exclude_vendored_libraries: Exclude code from vendored libraries from the index

    Returns:
        bool: True if index generation was successful, False otherwise
    """
    self.index_generation_report = None
    if not self._discard_generation_outputs():
        self.index_generation_report = self._generation_report(
            status="failed", complete=False, document_count=0
        )
        return False

    # Check if Cargo.toml exists
    cargo_toml = self.project_root / "Cargo.toml"
    if cargo_toml.is_symlink() or not cargo_toml.is_file():
        logger.error(
            f"Cargo.toml not found at {cargo_toml}. "
            "This doesn't appear to be a Rust project."
        )
        self.index_generation_report = self._generation_report(
            status="failed", complete=False, document_count=0
        )
        return False

    try:
        success = super().generate_index(
            config_path=config_path,
            exclude_vendored_libraries=exclude_vendored_libraries,
        )
    except Exception:
        self._discard_generation_outputs()
        self.index_generation_report = self._generation_report(
            status="failed", complete=False, document_count=0
        )
        raise

    document_count = self._usable_index_document_count()
    if success and not self.index_file.is_symlink() and self.index_file.is_file():
        self.index_generation_report = self._generation_report(
            status="complete", complete=True, document_count=document_count
        )
        return True

    self._discard_generation_outputs()
    self.index_generation_report = self._generation_report(
        status="failed", complete=False, document_count=document_count
    )
    return False

run_pipeline ¶

run_pipeline(
    output_file: str | None = None,
    skip_level: str | None = None,
    *,
    reset_profiler: bool = True,
    report_profile: bool = True,
    **kwargs
)

Run Rust pipeline while ignoring Python-specific kwargs.

Source code in codenib/scip_interface/scip_indexer_rust.py

def run_pipeline(
    self,
    output_file: Optional[str] = None,
    skip_level: Optional[str] = None,
    *,
    reset_profiler: bool = True,
    report_profile: bool = True,
    **kwargs,
):
    """
    Run Rust pipeline while ignoring Python-specific kwargs.
    """
    kwargs.pop("project_name", None)
    kwargs.pop("target_dir", None)
    kwargs.pop("cwd", None)
    kwargs.pop("allow_project_preparation", None)

    return super().run_pipeline(
        output_file=output_file,
        skip_level=skip_level,
        reset_profiler=reset_profiler,
        report_profile=report_profile,
        **kwargs,
    )

SCIPTypeScriptIndexer ¶

SCIPTypeScriptIndexer(
    project_root: str | Path,
    output_dir: str | Path | None = None,
    exclude_patterns: list | None = None,
    profiler: Profiler | None = None,
    decoder_backend: str | None = None,
)

Bases: SCIPIndexerBase

SCIP indexer for TypeScript and JavaScript projects.

Uses the scip-typescript tool to generate SCIP indices for TypeScript/JavaScript codebases.

Parameters:

Name	Type	Description	Default
`project_root`	`str \| Path`	Root directory of the TypeScript/JavaScript project	required
`output_dir`	`str \| Path \| None`	Directory to store output files (defaults to ~/.codenib/{project_name})	`None`
`exclude_patterns`	`list \| None`	List of patterns to exclude from indexing	`None`
`profiler`	`Profiler \| None`	Profiler instance for performance tracking	`None`
`decoder_backend`	`str \| None`	`"serial"` (default) or `"core"`.	`None`

Methods:

Name	Description
`generate_index`	Generate SCIP index for the TypeScript/JavaScript project.
`run_pipeline`	Run TypeScript pipeline with language-specific defaults.

Source code in codenib/scip_interface/scip_indexer_ts.py

def __init__(
    self,
    project_root: Union[str, Path],
    output_dir: Optional[Union[str, Path]] = None,
    exclude_patterns: Optional[List] = None,
    profiler: Optional[Profiler] = None,
    decoder_backend: Optional[str] = None,
):
    """
    Initialize the TypeScript SCIP indexer.

    Args:
        project_root: Root directory of the TypeScript/JavaScript project
        output_dir: Directory to store output files
            (defaults to ~/.codenib/{project_name})
        exclude_patterns: List of patterns to exclude from indexing
        profiler: Profiler instance for performance tracking
        decoder_backend: ``"serial"`` (default) or ``"core"``.
    """
    super().__init__(
        project_root=project_root,
        output_dir=output_dir,
        exclude_patterns=exclude_patterns,
        profiler=profiler,
        language="typescript",
        decoder_backend=decoder_backend,
    )

generate_index ¶

generate_index(
    project_name: str | None = None,
    infer_tsconfig: bool = False,
    yarn_workspaces: bool = False,
    pnpm_workspaces: bool = False,
    npm_workspaces: bool = False,
    patched_tsconfig: str | None = None,
) -> bool

Generate SCIP index for the TypeScript/JavaScript project.

Parameters:

Name	Type	Description	Default
`project_name`	`str \| None`	Project name (not used by scip-typescript)	`None`
`infer_tsconfig`	`bool`	Infer tsconfig for JavaScript projects without tsconfig.json	`False`
`yarn_workspaces`	`bool`	Enable Yarn workspaces support	`False`
`pnpm_workspaces`	`bool`	Enable pnpm workspaces support	`False`
`npm_workspaces`	`bool`	Enable npm workspaces support	`False`
`patched_tsconfig`	`str \| None`	Path to a patched tsconfig file to use	`None`

Returns:

Name	Type	Description
`bool`	`bool`	True if index generation was successful, False otherwise

Source code in codenib/scip_interface/scip_indexer_ts.py

def generate_index(
    self,
    project_name: Optional[str] = None,
    infer_tsconfig: bool = False,
    yarn_workspaces: bool = False,
    pnpm_workspaces: bool = False,
    npm_workspaces: bool = False,
    patched_tsconfig: Optional[str] = None,
) -> bool:
    """
    Generate SCIP index for the TypeScript/JavaScript project.

    Args:
        project_name: Project name (not used by scip-typescript)
        infer_tsconfig: Infer tsconfig for JavaScript projects without tsconfig.json
        yarn_workspaces: Enable Yarn workspaces support
        pnpm_workspaces: Enable pnpm workspaces support
        npm_workspaces: Enable npm workspaces support
        patched_tsconfig: Path to a patched tsconfig file to use

    Returns:
        bool: True if index generation was successful, False otherwise
    """
    # Guardrail: if no explicit ts/js config exists, force infer mode.
    has_tsconfig = (self.project_root / "tsconfig.json").exists()
    has_jsconfig = (self.project_root / "jsconfig.json").exists()
    if not infer_tsconfig and not has_tsconfig and not has_jsconfig:
        infer_tsconfig = True
        logger.info(
            "No tsconfig.json/jsconfig.json in %s; forcing infer_tsconfig=True",
            self.project_root,
        )

    return super().generate_index(
        project_name=project_name,
        infer_tsconfig=infer_tsconfig,
        yarn_workspaces=yarn_workspaces,
        pnpm_workspaces=pnpm_workspaces,
        npm_workspaces=npm_workspaces,
        patched_tsconfig=patched_tsconfig,
    )

run_pipeline ¶

run_pipeline(
    output_file: str | None = None,
    skip_level: str | None = None,
    *,
    reset_profiler: bool = True,
    report_profile: bool = True,
    **kwargs
)

Run TypeScript pipeline with language-specific defaults.

If neither tsconfig.json nor jsconfig.json exists, enable --infer-tsconfig automatically unless caller explicitly sets it.

Source code in codenib/scip_interface/scip_indexer_ts.py

def run_pipeline(
    self,
    output_file: Optional[str] = None,
    skip_level: Optional[str] = None,
    *,
    reset_profiler: bool = True,
    report_profile: bool = True,
    **kwargs,
):
    """
    Run TypeScript pipeline with language-specific defaults.

    If neither tsconfig.json nor jsconfig.json exists, enable
    --infer-tsconfig automatically unless caller explicitly sets it.
    """
    # Drop Python-specific kwargs that may be forwarded by callers.
    kwargs.pop("target_dir", None)
    kwargs.pop("cwd", None)
    allow_project_preparation = kwargs.pop("allow_project_preparation", True)
    if type(allow_project_preparation) is not bool:
        raise ValueError("allow_project_preparation must be a boolean")

    # Auto-select workspace mode when not explicitly provided.
    workspace_flags = ("yarn_workspaces", "pnpm_workspaces", "npm_workspaces")
    has_workspace_mode = any(kwargs.get(flag) for flag in workspace_flags)
    has_root_config = (self.project_root / "tsconfig.json").is_file() or (
        self.project_root / "jsconfig.json"
    ).is_file()
    if not has_workspace_mode and not has_root_config:
        # A root config is an explicit project boundary and may already
        # include every workspace source. Passing a workspace flag instead
        # makes scip-typescript enumerate package manifests and can turn a
        # healthy root project (for example Vue) into a successful but
        # nearly empty index when its member packages have no tsconfig.
        # pnpm defines workspaces in pnpm-workspace.yaml (not package.json),
        # so check for it independently.
        has_pnpm_workspace_yaml = (
            self.project_root / "pnpm-workspace.yaml"
        ).exists()

        has_pkg_workspaces = False
        package_json = self.project_root / "package.json"
        if package_json.exists():
            try:
                payload = json.loads(package_json.read_text(encoding="utf-8"))
                has_pkg_workspaces = bool(payload.get("workspaces"))
            except Exception:
                pass

        if has_pnpm_workspace_yaml and shutil.which("pnpm"):
            kwargs["pnpm_workspaces"] = True
            logger.info("Detected pnpm workspace in %s", self.project_root)
        elif has_pkg_workspaces:
            if (self.project_root / "yarn.lock").exists() and shutil.which("yarn"):
                kwargs["yarn_workspaces"] = True
                logger.info("Detected yarn workspace in %s", self.project_root)
            else:
                kwargs["npm_workspaces"] = True
                logger.info("Detected npm workspace in %s", self.project_root)

    # Only enable --infer-tsconfig when no tsconfig/jsconfig exists.
    if "infer_tsconfig" not in kwargs:
        has_tsconfig = (self.project_root / "tsconfig.json").exists()
        has_jsconfig = (self.project_root / "jsconfig.json").exists()
        if not has_tsconfig and not has_jsconfig:
            kwargs["infer_tsconfig"] = True
            logger.info(
                "No tsconfig.json/jsconfig.json; enabling infer_tsconfig for %s",
                self.project_root,
            )
        else:
            kwargs["infer_tsconfig"] = False
            logger.info(
                "Using existing tsconfig/jsconfig for %s", self.project_root
            )

    kwargs = self._normalize_workspace_kwargs(kwargs)

    # Determine whether index generation will actually run.
    # and dependency installation / tsconfig patching
    needs_generate = True
    if skip_level == "graph" and self.graph_file.exists():
        needs_generate = False
    elif skip_level in ("graph", "decode") and self.decoded_file.exists():
        needs_generate = False
    elif skip_level in ("graph", "decode", "raw") and self.index_file.exists():
        needs_generate = False

    cleanup_patched_tsconfig = False
    read_only_tsconfig: Optional[Path] = None
    if needs_generate:
        if allow_project_preparation:
            self._install_dependencies()
            patched_tsconfig = self._ensure_allow_js()
            if patched_tsconfig is not None:
                kwargs["patched_tsconfig"] = str(patched_tsconfig)
                cleanup_patched_tsconfig = True
        else:
            logger.info(
                "Skipping project dependency installation for read-only "
                "CodeGraph onboarding"
            )
            if self._read_only_config_needs_wrapper():
                read_only_tsconfig = self._create_read_only_tsconfig()
                kwargs["patched_tsconfig"] = str(read_only_tsconfig)
                kwargs["infer_tsconfig"] = False
    else:
        logger.info(
            "Skipping dependency install and tsconfig patching (cached artifacts exist)"
        )

    logger.info("TypeScript run_pipeline kwargs: %s", kwargs)
    try:
        return super().run_pipeline(
            output_file=output_file,
            skip_level=skip_level,
            reset_profiler=reset_profiler,
            report_profile=report_profile,
            **kwargs,
        )
    finally:
        if cleanup_patched_tsconfig:
            self._cleanup_patched_tsconfig()
        if read_only_tsconfig is not None:
            read_only_tsconfig.unlink(missing_ok=True)

codenib.scip_interface ¶

SCIPLocation dataclass ¶

SCIPOccurrence dataclass ¶

SCIPOccurrenceIndex ¶

from_scip_index classmethod ¶

load_stream classmethod ¶

SCIPCSharpGraphDecoder ¶

SCIPJavaGraphDecoder ¶

SCIPPHPGraphDecoder ¶

SCIPPythonGraphDecoder ¶

decode ¶

save_graph ¶

SCIPRubyGraphDecoder ¶

SCIPRustGraphDecoder ¶

decode ¶

save_graph ¶

SCIPTypeScriptGraphDecoder ¶

SCIPIndexerBase ¶

generate_index ¶

decode_index ¶

process_index ¶

run_pipeline ¶

clear_cache ¶

SCIPCSharpIndexer ¶

SCIPJavaIndexer ¶

SCIPKotlinIndexer ¶

SCIPScalaIndexer ¶

PHPHybridIndexer ¶

SCIPPHPIndexer ¶

SCIPPythonIndexer ¶

generate_index ¶

process_index ¶

run_pipeline ¶

RubyHybridIndexer ¶

SCIPRubyIndexer ¶

SCIPRustIndexer ¶

generate_index ¶

run_pipeline ¶

SCIPTypeScriptIndexer ¶

generate_index ¶

run_pipeline ¶

SCIPLocation `dataclass` ¶

SCIPOccurrence `dataclass` ¶

from_scip_index `classmethod` ¶

load_stream `classmethod` ¶