Skip to content
small logo large logo

Early Findings from an A.I. Writing Assistant Pilot

Early Findings from an A.I. Writing Assistant Pilot

At the National Institute for Student Success (NISS), we are continually exploring scalable ways to extend high-quality academic support beyond the classroom.  In partnership with Georgia State University and Studiosity, we piloted an AI-enabled writing feedback tool to better understand how timely, iterative feedback can support student learning. Studiosity is an academic support platform that helps students strengthen their writing, critical thinking, and learning skills through a combination of AI-enabled tools and human oversight. Integrated into an institution’s learning management system, the platform delivers personalized feedback and formative guidance while helping institutions support student success, validate learning, and promote academic integrity.

Pilot Design and Implementation

In Spring and Fall 2025, Georgia State faculty across multiple disciplines integrated Studiosity into their courses. The expanded fall pilot included 18 faculty and 1,168 undergraduate students.

Student Engagement Patterns

The pilot demonstrated clear demand for flexible, immediate support. Across the study period, the platform generated more than 3,400 student interactions delivered to 942 students, demonstrating sustained use over time. Student feedback further underscored the accessibility of the tool, with 92 percent reporting that the platform was easy to use.

The timing of student engagement was particularly noteworthy. More than 80 percent of engagement occurred outside of the traditional 9 a.m.–5 p.m. window with late evening (11 p.m.) and Sundays as peak usage periods. These findings reinforce a core principle of the NISS student success model: institutions achieve stronger outcomes when support structures are designed with students’ lives and behaviors in mind rather than institutional schedules. AI-enabled feedback is an example of operationalizing this principle. The high levels of evening and weekend engagement demonstrate how technology can help institutions extend support in ways that are responsive to the realities many students face, particularly those balancing employment and family obligations.

Evidence of Writing Improvement

Students were not simply using the platform, they appeared to be applying the feedback they received. Among evaluated writing samples, 66.7 percent showed improvement across drafts, indicating that many students were able to incorporate feedback into revisions. Faculty feedback further noted improvements in clarity, organization, and sentence-level accuracy, particularly in assignments structured around iterative drafting.

Academic Impact and Implications

Perhaps most striking were the differences in course outcomes. Across participating courses, faculty observed increased student confidence and self-revision behaviors alongside substantially lower DFW rates. Students who engaged with Studiosity had a DFW rate of 8.1%, compared with 38.9% among non-users, a difference of 30.8 percentage points. Although participation was voluntary and the results may reflect some degree of self-selection bias, these patterns suggest that access to timely feedback may support stronger course performance.

These early findings also align with NISS’s broader approach to student success, which emphasizes providing timely, actionable support before students encounter significant academic setbacks. By creating additional opportunities for feedback and revision, the tool may help institutions build the kind of proactive support ecosystem associated with stronger student outcomes. More broadly, these findings illustrate how technology can be leveraged to advance the proactive support principles that underpin the NISS’s approach to student success.

Conclusion

Findings from this pilot suggest that AI-enabled writing feedback tools can extend instructional assistance in meaningful ways, particularly by reaching students outside traditional hours and across diverse course contexts. These early results indicate that adaptive, on-demand feedback powered by A.I. may represent a promising strategy for improving outcomes at scale.

More broadly, the pilot illustrates how institutions can leverage technology in service of a core NISS principle: delivering the right resources to students at the right time and in the right way. Consistent with the Georgia State and NISS approach to student success, the goal is not simply to offer additional services but to design systems that proactively connect students with the guidance they need when they need it most.

Rather than replacing faculty or existing student services, AI-enabled tools can help institutions scale evidence-based practices and extend their reach to students who might otherwise go unserved. As colleges continue to seek ways to improve outcomes, these findings provide another example of how intentional, student-centered design can strengthen success at scale.