Complex sensing and decision applications such as object tracking and classification, video surveillance, unmanned aerial vehicle flight decisions, and others operate on vast data streams with dynamic characteristics. As the availability and quality of the sensed data changes, the underlying models and decision algorithms should continually adapt in order to meet desired high-level requirements. Due to the complexity of such dynamic data-driven systems, traditional design time techniques are often incapable of producing a solution that remains optimal in the face of dynamically changing data, algorithms, and even availability of computational resources. To assist developers of these systems, we present a modeling and optimization methodology that enables developers to capture application task flows and data sources, define associated quality metrics with data types, specify each algorithm's data and quality requirements, and define a data quality estimation framework to optimize the application at runtime. We demonstrate each facet of the modeling and optimization process via a video-based vehicle tracking and collision avoidance application, and show how such an approach results in efficient design space exploration when selecting the optimal set of algorithm modalities. When searching for an application configuration within 1% to 5% of optimal, our model-guided approach can achieve speedups of up to 9.3X versus a standard genetic algorithm and speedups of up to 80X relative to a brute force algorithm.