Detecting Behavioral Changes in Python Refactoring Implementations with Foundation Models

Investigating the Refactoring Capabilities of Small Open-Weight Language Models

ML-Enhanced Code Refactoring Recommendations for Improving Software Maintainability

Results show that ML-enhanced recommendations outperform traditional methods in accuracy, relevance, and impact on maintainability metrics, and highlight the potential of integrating ML into modern development practices to support developers in producing cleaner, more maintainable software systems.

When Testing Meets Refactoring: Catalogue, Detection, and Recommendation

This paper presents a unified catalogue of test refactoring patterns derived from a mixed-method approach involving repository mining, developer surveys, and analysis of StackOverflow discussions, to establish a rigorous benchmark for evaluating future automated recommendation systems, including those driven by Large Language Models.

RefactorAssist: Agentic Refinement for Reliable Code Refactoring

Results indicate that static checks and test-guided, context-aware agentic repair can increase the reliability of LLM-generated refactorings, bringing them closer to practical integration within developer workflows.

RepoProbe: Benchmarking Architecture-Aware Repository Comprehension with Checklists

The integration of Large Language Models (LLMs) into software engineering has shifted the focus from function-level generation to repository-scale assistance. However, existing benchmarks largely rely on bug reports from GitHub Issues, which often allow models to bypass genuine understanding via pattern matching on error logs. This misalignment under-measures Edit Bias, which refers to premature generation, where models prematurely propose code modifications instead of understanding the existing repository architecture. Furthermore, current LLM-as-a-Judge scalar scoring suffers from high variance and low interpretability. This work introduces RepoProbe, a novel benchmark for evaluating repository-level code understanding through open-ended Q&A using GitHub Discussions, which focuses on open-ended architectural inquiries rather than defect reporting. To ensure rigorous evaluation, we propose a Checklist-Based Verification Protocol that decomposes answers into atomic, verifiable facts, thereby replacing subjective ratings with objective verification. Our evaluation of state-of-the-art (SOTA) LLMs reveals a persistent gap between high clarity and evidencegrounded technical correctness. It also quantitatively confirms the prevalence of edit bias, in which models prioritize code generation instead of architectural analysis. Finally, we demonstrate that our verification protocol significantly improves evaluation reliability compared to traditional evaluations with scalar scoring.

PyMOP: A Runtime Verification Tool for Python

Runtime verification (RV) now scales for testing in thousands of open-source Java projects, helping find hundreds of bugs by monitoring test executions against formal specifications (specs). The popular Python ecosystem could use such benefits. But, current Python RV tools are limited to a domain or spec logic, or they are slow. We present PyMOP, a generic, extensible, and more efficient Python RV tool. PyMOP supports five logics, implements five monitoring algorithms, ships with 81 specs, and supports three instrumentation strategies. On 48,090 unit tests in 839 GitHub projects, we find mainly that (i) PyMOP is up to 419.23x faster than two recent dynamic analysis tools; (ii) Expensive instrumentation is a main cause of RV's runtime overhead for Python; and (iii) 84 of 156 bugs that PyMOP found were confirmed or fixed by developers. PyMOP's genericness and efficiency position it well as an excellent platform for the next advances on RV for Python. PyMOP is at https://github.com/SoftEngResearch/pymop and a video demo is at https://pymop.zhuohangshen.com/demo.