For many algorithms, parameter tuning remains a challenging task, which be- comes tedious in a multi-parameter setting. Multi-penalty regularization, suc- cessfully used for solving undetermined sparse regression problems of unmixing type, is one of such examples. We propose a novel algorithmic framework for an adaptive parameter choice in multi-penalty regularization with focus on correct support recovery. By extending ideas on regularization paths, we provide an efficient procedure for the construction of regions containing structurally sim- ilar solutions, i.e., solutions with the same sparsity and sign pattern, over the range of parameters. Combined with a model selection criterion, regularization parameters are chosen in a data-adaptive manner. Another advantage of our algorithm is that it provides an overview on the solution stability over the pa- rameter range. We provide a numerical analysis of our method and compare it to the state-of-the-art algorithms for compressed sensing problems to demonstrate the robustness and power of the proposed algorithm.