---

From left:
- Iori Kinoshita, Information Systems Team
- Toshio Fujiwara, Head of Development Department
- Nozomi Ito, MagicPod CEO

---

Iori: In my previous role, I spent about three years as a tester handling test execution and test design. I joined 4COLORS in April 2025, taking over QA responsibilities from my predecessor during a migration from the old system to a new one. Since MagicPod's implementation happened to be decided right around that time, I ended up taking on test automation responsibilities in addition to my regular testing work.
Our PIP-Maker service allows users to create avatar and voiceover videos in as little as five minutes simply by uploading a PowerPoint file. It makes video production — which previously required significant cost — accessible to anyone, quickly and affordably, for use cases like e-learning, product introductions, and manual creation.

---

I handle quality assurance for PIP-Maker, covering everything from validating new features to checking for display issues caused by OS and browser updates, and verifying video playback stability. The Information Systems Team has three members in total, but I am the only one responsible for quality control on PIP-Maker. I essentially cover all verification work on my own, and currently run automated tests in MagicPod every day.

Toshio: I joined the company in 2019 and currently serve as Head of the Development Department, overseeing team management and acting as PM for PIP-Maker. The Development Department has five members on the books, but the total number of people involved in PIP-Maker development and product work on a project basis is around ten.

We originally handled all testing manually, and introduced MagicPod with the goal of automating quality control. Last year in particular, we were in the middle of migrating from the old system to the new one, and we used that as an opportunity to shift to a more efficient testing approach — conducting manual testing for new features and other verification, while automating daily monitoring and regression testing.

---

Why We Chose MagicPod

Nozomi: Before the QA function was established within the Information Systems Team, how were you handling quality control?

Toshio: At the time, members of our internal support team were doing a minimum level of external monitoring — manually checking every morning to confirm the service hadn't gone down. There was already a general sense internally that we should automate as much as possible, but the core challenge was: how do we reduce testing man-hours while still maintaining quality?

PIP-Maker converts slides into video by nature, which means there's a large volume of visual verification items — not just UI button operations, but things like "is the video rendering correctly?" and "are the avatars and text positioned without layout issues?" We had been manually playing back each video one by one to verify, but checking every combination of OS and browser was approaching its limits. The risk of human error and the growing testing man-hours for regression testing had become significant challenges.

In principle, the development team recognized that we should have been building and running our own E2E testing infrastructure, but the reality was that we simply couldn't allocate the man-hours. Seeing this situation, my predecessor started looking for an automation tool around 2024.

Nozomi: Before you introduced MagicPod, were there cases where defects caused by missed checks actually surfaced?

Iori: Based on what I heard from my predecessor, there were apparently a few close calls. For example, I was told there were several cases in the past where a function stopped working as expected after a release, and it was caught internally before customers noticed. The manual verification scope had limits, and there were always gaps in what could be checked.

From those experiences, the push for automation began — the goal being to move away from "we happened to catch it internally" toward "we want to monitor consistently and at a high level of accuracy every day."

When evaluating options, it sounds like they compared various no-code tools from Japan and overseas, as well as Selenium. In that process, they found MagicPod, tried it out, and concluded it was the best option — so they moved forward with adoption. Since the handover to me happened right after the selection was finalized, I was already involved from the trial stage before the official implementation.

Toshio: The deciding factors in choosing MagicPod were the low barrier to entry and the cost performance.

With no time available for development engineers to write test code, MagicPod allows tests to be created and executed intuitively with no-code, meaning operations could begin immediately with almost no learning cost. Another appeal was its flexibility to handle the kinds of specialized tests unique to our company — not just screen button operations, but tests that reproduce user viewing behavior to verify whether viewing logs are being accumulated correctly.

Given the nature of our product, the highest-impact issue for users would be "not being able to watch a video," followed by "not being able to edit a video." These two are overwhelmingly the most critical areas, so the core functions — "can videos be watched without issues?" and "can videos be edited reliably?" — are what we currently have automated and running.

---

---

Using the Test Automation Launch Support Plan

Nozomi: You used the Launch Support Plan when implementing MagicPod — how was that experience?

Iori: At the time, I was in the middle of the handover from my predecessor and had a heavy workload, so I felt that learning a new tool entirely on my own would be a significant burden and decided to ask for support. I had already started building tests before the plan began, but having the practical, applied know-how explained to me in detail through the support plan allowed me to smoothly transition into a daily automated test operation.

We started with foundational concepts — what is XML, what is XPath — and then moved into a hands-on format where I could get direct guidance on parts I had built but couldn't get working. I could also request topics ahead of the next session, such as "I'd like to learn more about this feature," which meant the time was structured around two axes: resolving things I didn't understand, and picking up new capabilities. It was a very effective use of time.

Without the support plan, I think I would have spent an enormous amount of time trying to understand the specs and troubleshoot on my own. Having the underlying logic explained in detail meant I understood how tests actually work at a fundamental level — and from there I could think, "if this works this way, then I should be able to apply it like this next." Even for simple tests I can now build on my own, the deep understanding I gained through this plan has been the foundation when I need to build something more precisely to my specific requirements.

Nozomi: When did you feel genuinely confident that you could run tests on your own?

Iori: The support plan wrapped up after about 15 hours over 3–4 months, and by that point I had a solid grasp of how to apply things. When someone internally said "I'd like to try this kind of test," I was at a level where I could immediately think of a specific approach — "I could combine those features in that way to build it."

At first, honestly, 15 hours felt like a long time. But as my understanding grew, the questions I wanted to ask kept multiplying. We progressed through a very clean cycle: building foundational understanding in the early sessions, clearing up day-to-day questions in the middle, and resolving more advanced topics toward the end. In hindsight, 15 hours was exactly the right length.

Nozomi: From the development side, how did you perceive Iori's skill development and the sense of "having let go"?

Toshio: She has been operating independently at a level that far exceeded our expectations. She has built tests that genuinely surprised me — I hadn't anticipated such a sophisticated automation setup initially, and I found myself thinking, "MagicPod can really do this much." The work has fully left our hands on the development side, and we can confidently rely on her for high-quality verification. I really believe we made the right call in using the support plan.

---

---

How MagicPod Is Being Used

Iori: We have automated everything for PIP-Maker, from operations on the video editing screen through to preview playback of the generated video. When a specific slide is converted into video, the system automatically checks whether avatars and graphics are positioned and displayed as expected. In the production environment, we run basic operation tests including peripheral features every day for early defect detection, and in the development environment we run broader, scheduled tests on a weekly basis. We currently operate a total of 32 test cases.

In terms of specific execution timing, we run a full feature check in Chrome twice daily — once around 8:00 AM before the workday begins, and once before end of day. We also have the same tests running in Edge and Firefox during late-night hours. Separately from those, we run a one-hour self-contained test every hour around the clock that monitors video playback every 3 minutes, providing continuous 24/7 coverage.

Nozomi: Is the every-3-minute test running stably?

Iori: It took some effort at the start, but after iterating on the setup it now runs very reliably. When a video playback issue occurs, it's detected immediately, and the notification is sent to a company-wide channel so the relevant person can be contacted right away when needed.

Nozomi: That's an impressive operation. Overall, how far along would you say the automation of your testing is at this point?

Iori: For the core areas we want checked automatically every day, we've reached a state that's close to 100%. That said, the product's features are continuously being updated, so there's no true "100% done." As I've been with the company longer and my understanding of the product deepens, new ideas for tests keep emerging — "we should probably have a test for this too." So I'd put overall progress at around 70% at this point.

Toshio: The significance of being able to verify — through MagicPod — whether users can actually watch videos properly is enormous. Since MagicPod itself is stable, I'm currently comfortable leaving baseline monitoring in Iori's hands.

Nozomi: You're running scheduled tests across multiple time slots and browsers — so you're also monitoring the impact of browser updates?

Toshio: Exactly. Chrome in particular has frequent unexpected specification changes, and in the past there were incidents where a spec change caused videos to become unwatchable. By running scheduled tests regularly in the development environment as well, our ability to detect issues during browser updates has improved dramatically. Beyond the obvious quality benefit, the psychological peace of mind — "we can prevent playback issues before they happen" — has taken hold across the team, and being able to catch regressions early during development has been a major outcome as well.

Iori: There's also been a significant impact in terms of reducing testing man-hours. We have releases almost every week, and previously regression testing done manually would take about 4 hours. With automation, it now takes around 2 hours.

Beyond that, while MagicPod is running, I can now perform more detailed manual testing in parallel. Running manual and automated testing simultaneously has allowed us to cut the actual time commitment in half while substantially expanding test coverage.

Nozomi: It sounds like you've found a very effective rhythm of balancing daily automated tests with manual verification. I also heard you're running quite advanced test operations using data patterns and variables — can you walk us through that?

Iori: We actually use MagicPod not just for screen testing, but as a tool for accumulating viewing data.

PIP-Maker is built to collect data in the background on who watched a video, when, and how far they got. Customers who use it for internal training, for example, use that data to check whether attendees understood the content through quizzes, or to follow up with people who haven't completed the training yet.

For that reason, we need to internally verify the report feature — specifically, how the system performs when customers generate large-scale viewing reports from the admin dashboard. Originally, preparing the necessary data required manually playing back videos many times to accumulate the required number of records, and there were limits to how much data could be realistically prepared this way. Solving this — building a mechanism to automatically pre-accumulate logs — was something I wanted to achieve from the very beginning of the implementation.

So now, every day MagicPod automatically accesses videos and uses carefully combined data patterns and variables to generate realistic viewing logs. For example: if a user ID ends in 2 or 5, the behavior is set to "exit the video partway through." For in-video quizzes and surveys, the answer selection depends on the viewing time at that moment — even-numbered minutes select Option A, odd-numbered minutes select Option B. Through these conditional branches, a single test case can efficiently collect varied, natural-looking viewing data in the backend — without it being uniform or artificial.

This initiative started with gradual experimentation around August 2025, and by around autumn I was able to combine data patterns and variables freely and operate the system confidently. Being able to ask specific questions during the Launch Support Plan sessions — on topics like "how to embed variables within data patterns" — was what made it possible to turn the idea into reality. That led to a deeper understanding of how to build far more diverse patterns.

Nozomi: Are you also using the AI features?

Iori: Yes — I use MagicPod Autopilot when it's time to update test cases. Recently there was a fairly large UI change, and for that I manually corrected the first instance myself, then instructed Autopilot to "rewrite all elements like this one in the same way," and it handled the bulk update. I expect the range of uses to expand even further as we accumulate more operational know-how.

I've also been finding the AI failure analysis feature very useful lately. When a test fails due to an unexpected alert screen or similar issue, it can be difficult to trace the technical root cause all the way down to the code level on my own — but I can't report a defect to the development team with an unclear cause. With the failure analysis AI, it explains things clearly — "this failure occurred because an alert with this content was displayed" — which has made defect reporting to the development side much smoother. I've genuinely felt the power of AI not just in fixing things, but in making the reporting process more efficient as well.

---

---

Closing

Iori: Right now we're focused primarily on regression testing by feature, with verification across a variety of data patterns. Going forward, I'd like to build out "user scenario tests" that trace the full flow from initial account registration — tests that simulate real customer behavior end to end. Until now I've focused heavily on individual feature verification, so I want to shift toward more scenario-based tests and continue raising the quality bar.

For non-engineers taking on test creation, there will likely be moments early on where things don't work as expected or the root cause isn't clear. But MagicPod offers a very thorough and thoughtful support structure, and the Launch Support Plan covers everything from basics to advanced application. Going in with the mindset of "let's try building it ourselves first, and actively ask questions when we hit a wall" means even someone with no programming experience can move forward with test automation confidently. The ability to reach out for support easily and with a low psychological barrier — right from within the interface — is another thing I'd highlight.

Toshio: On the development side, we're looking to strengthen CI/CD integration. Since our current operations are primarily centered on the production environment, we want to move toward running automated tests earlier in the verification environment so that the development team can take ownership of defect detection.

For companies where development engineers don't have the bandwidth to write test code, MagicPod delivers significant cost-effectiveness — covering this much test coverage at this price point. The solid sense of reassurance it creates within the team — "as long as we're running these regression tests, we have a reasonable level of quality covered" — is a major mental benefit that genuinely helps accelerate development.

---

4COLORS Co., Ltd.

• Product site：https://www.pip-maker.com/
• Corporate site：https://www.4colors.jp/

---

---

From left:
Tetsuya Kawase — Technology Division, Product Development Group, Vista Department
Nozomi Ito — MagicPod CEO

---

Kawase: I joined Hacobu in August 2023. I am responsible for QA on the dispatch order and management service "MOVO Vista." Hacobu does not have a centralized quality assurance department that spans the entire organization. Instead, we have built a "Product QA" structure in which a dedicated QA professional is embedded in each product team to develop and execute the optimal test strategy for that product.

With this structure, the consensus-building needed to launch new initiatives is handled entirely within the team, which means QA decisions directly shape the test strategy and quality improvement efforts for the whole team. I find it easier to move forward with genuine buy-in, and the agility it enables is a significant advantage. We have nine full-time QA engineers, each operating independently, and we hold a sharing session roughly every two weeks. We also have active informal groups — similar to guilds — where members with shared interests come together in small groups to work on things collaboratively.

I originally started my career as a software engineer, spending about 14 years at an SES company working on system development for other organizations. In the latter part of that period, I increasingly moved into roles overseeing entire teams and projects, and on smaller engagements I had more and more opportunities to handle scenario testing and integration tests.

After that, I joined another company in a business planning and SI vendor liaison role, and became involved in an app development initiative the company was running at the time. Through that experience, I discovered the interest of engaging not only with the builder's perspective of quality — "making sure the system works" — but also with quality from a user's point of view: "what does it take for customers to be genuinely satisfied?" That sparked my interest in the field, and after gaining experience at a third-party verification firm, I joined Hacobu.

---

Reasons and Background for Selecting MagicPod

Kawase: Product QA is one of Hacobu's strengths, but it also carries the risk of over-reliance on individual knowledge. When I joined, the company was in a phase that prioritized the speed of agile development, and regression testing had not been properly established — the prevailing understanding was that manual testing would cover what it could. The judgment of which tests to run depended on the discretion of each individual, and the situation was such that incidents like "there's a regression somewhere but we can't identify the cause" could easily arise.

Ito: Was it you who drove the adoption of automated testing and the selection of a specific tool?

Kawase: It had already been under consideration before I joined. It seemed to be a concern at the leadership level as well, and when MagicPod was introduced, our CEO responded positively. After I joined, the evaluation came down to a choice between a recording-type tool and MagicPod. In that process, we assessed MagicPod favorably for its straightforward creation flow and high maintainability. Personally, I also appreciated that Shared Steps were available.

Incidentally, I first came across MagicPod at my previous job at a third-party verification firm, when a client company was exploring automation of their regression testing. Another person was leading that effort so my involvement was limited to trying it out, but they ultimately adopted it there as well.

Ito: Many companies appreciate the ability to run tests without limits — was that not a significant factor for you?

Kawase: At the time, I don't think I had a concrete enough picture to fully appreciate it. That said, hearing you mention it now, being able to run tests every day without hesitation is genuinely a major benefit.

Ito: Thank you! Were overseas testing tools or Selenium also among the candidates?

Kawase: In the earlier rounds of evaluation, yes, those were on the list. However, recording-type tools — which are common among overseas products — tend to depend heavily on how each individual builds tests, making it difficult to maintain consistent quality. Coding-based tools, on the other hand, raised concerns about the onboarding learning curve, since programming experience varied across team members, and so they were ruled out. Ultimately, we selected MagicPod for its ability to support structured test creation and its high maintainability.

Ito: With product teams operating separately under a Product QA structure, I imagine it can be difficult to drive adoption across the organization. How did you approach that?

Kawase: Initially there was another person driving the effort alongside me, and that person first introduced MagicPod in the truck appointment scheduling service "MOVO Berth." Through that process, we established a full set of review mechanisms, review criteria, and test creation guidelines, and when rolling out to other products, we applied those frameworks while continuing to refine them.

I believe that in test automation, what matters more than the tool itself is designing the structure through which quality is assured. It is often misunderstood that "because it's no-code, no design is necessary" — but my view is that "no-code ≠ no design."

In practice, when teams jump straight into implementation without sufficiently thinking through the test structure, data handling, and rule design — the "architecture" of automation — changes become difficult to make later, and the result is often what I would call a hollowing-out of the automation: failed tests get left unresolved, and the whole effort falls into disuse.

At the same time, the mechanics of how to use the tool itself can be picked up along the way through hands-on practice. That is precisely why our team prioritized getting the hard-to-change, low-leverage-once-set elements — test structure, data design, and operational rules — sorted out before we began implementation.

---

Foundation Design in the Early Stages of MagicPod Implementation

Ito: What specifically did that preparation involve?

Kawase: In software development, coding standards and variable naming conventions are typically in place to ensure a baseline of quality. Drawing on my own background as an engineer, I applied a similar approach to test automation and organized the groundwork into four broad layers.

The first was structural design, with post-implementation maintainability and ease of implementation in mind. By clarifying which MagicPod features map to which traditional test structure concepts, the team was able to develop shared understanding and move forward with implementation on a common foundation.

---

Next, we established guidelines and naming conventions to enable safe Data Driven Testing. Naming rules are differentiated by variable type — for example, shared variables use uppercase snake_case, while variables within test cases use camelCase.

---

The third layer was operational design. Establishing structure, guidelines, and conventions alone is not enough to ensure they are actually followed, so we documented the implementation flow for test cases, the review flow, and the process from investigating a test failure through to resolution.

Finally, we prepared a "usage guide." In the early days after implementation, test case creation was a process of trial and error, so we set up a page to accumulate observations and anti-patterns from implementation, allowing the team to share good practices and patterns to avoid. Through this preparation, I believe we were able to build a foundation where operations would not become hollow over time and where adjustments would remain manageable.

Ito: That is impressive. It is rare for an organization to have this level of preparation in place before introducing a tool. I felt that the "no-code ≠ no design" philosophy is genuinely reflected in how you operate.

Kawase: Thank you. That said, maintaining this operational foundation over the long term also required keeping the team motivated in the early stages. What proved helpful there was MagicPod's analytics feature. In particular, we set stabilizing the "health score" — which measures the health of operations — in the 90s as our near-term goal. Because the evaluation criteria are broken down into incremental stages with the importance and priority of each item displayed, it was easy to set short-term milestones, and I felt that contributed to sustaining motivation.

---

How MagicPod Is Being Used

Kawase: Currently, we have automated testing for MOVO Berth, MOVO Vista, and a third — including a smartphone app — bringing the total to three products covered by MagicPod. Accounts have been issued to developers as well, bringing the total to around 50 users. Depending on the team, in the product I am involved in, MagicPod is used as part of the release approval process, and there are cases where developers independently run MagicPod outside the regular release schedule to make their own release judgments.

Scheduled Execution kicks off at 7:30 in the morning and completes in about 45 minutes, with results delivered to a dedicated Slack channel that only receives execution results. In general, when a Batch run test fails, QA investigates — but members who are engaged will proactively look into the error on their own. They first determine whether the issue is transient, and if it turns out to be a defect on the frontend or backend, a fix is applied and the release approval run is executed again.

There was actually a point in the past where production releases were going out before the daily test results had been reviewed, and we had a problem with testing and releases being disconnected. So we established a clear release standard: "all morning tests must have passed." For runs outside that window as well, we use a workflow automation tool — a Slack command triggers a Batch run automatically.

Ito: It sounds like your development engineers are also engaging with MagicPod. Have there been any positive responses since the rollout?

Kawase: We have received positive feedback about the fact that defects that previously slipped through testing are now being caught before release. There is a shared understanding across the entire team that proper verification happens before a release goes out, and I feel that the resulting sense of assurance is a visible effect. Beyond that, incorporating MagicPod into the release approval process has led to engineers proactively asking "Did you run MagicPod?" — and a culture has emerged where the team locks down release content by the day before to make it in time for the morning tests. That improvement in quality awareness across the whole organization has been one of the most significant outcomes.

Ito: It is wonderful to see such a healthy culture taking root across the development team. For that kind of stable daily operation to be sustainable, the maintainability of the test cases themselves also becomes important. Do you have any criteria for granularity when creating Shared Steps, or any tips for improving maintainability?

Kawase: As a baseline, we always implement API calls as Shared Steps — handling everything from the call itself through to result retrieval and return as a single unit. Beyond that, our policy is to actively consolidate anything that is used frequently or corresponds to shared UI components on the frontend.

Another practice we use to improve maintainability is an automated review system leveraging AI. Specifically, we integrate Cursor with MagicPod's MCP. We preload all of our coding rules and variable naming conventions into Cursor's configuration and created a custom command from there.

When you run this command and specify a test case number, the AI automatically performs a review against the conventions and feeds back the points that need to be corrected. In practice, each team member runs this automated review in their local environment, writes the output to Notion, and then uses that as the basis for requesting a final confirmation review from other team members. Once there are no issues, the test case moves into live operation.

Beyond reviews, we also run processing via Cursor through the MCP server to re-execute only the tests that failed in a Batch run, and to fetch data via the API and generate graphs showing trends in execution time. Aggregation and analysis tasks that would take more effort through the GUI can be executed simply by entering a natural language prompt, and I feel that has significantly expanded the freedom we have in managing test automation operations.

Ito: It sounds like quite a sophisticated setup — automated AI-powered reviews, aggregation via MCP, and more. Going forward, how are you thinking about rolling these practices out to other team members and teams within the company?

Kawase: Our organizational culture places a very strong emphasis on individual autonomy and team-level optimization. So rather than managing things top-down, the challenge going forward is how to expand these practices while leveraging the initiative of people on the ground. That said, one of Hacobu's strengths is a culture where proposing to try a new tool gets a "let's go for it" response from the company, and where lively discussion happens across team boundaries on Slack and elsewhere. I think the path forward is to leverage that environment to spread adoption organically.

---

Toward a QA Organization That Keeps Pace with the Speed of AI-Driven Development

Ito: What challenges are you hoping to address going forward with the help of AI?

Kawase: Proficiency with AI varies across teams and individuals. Some members struggle to effectively translate AI suggestions into test cases, while others find themselves in trial-and-error mode when unexpected side effects arise during fixes. There is also a significant challenge around not being fully aware in advance of which MagicPod test cases will be affected by a given change — even when those tests are part of the release approval criteria, the impact sometimes only becomes clear after the fact. That invisible blast radius is something I want AI to help us detect.

Ito: You mean identifying the scope of impact from a change. One approach that has emerged recently is to pass a GitHub pull request URL to an AI editor like Cursor via MagicPod MCP and instruct it to "list the tests that are likely to be affected by this change" — which allows for a reasonable degree of pre-run impact assessment without actually executing the tests.

Kawase: That sounds very useful — I'll look into it. Our development environment is quite complex, with dev and staging environments running alongside topic environments for each development unit. When resources allow, we can do comprehensive checks in the topic environment, but when things get busy, there are inevitably cases where verification ends up being limited to the dev environment.

Inheriting tests created by someone else is particularly challenging — it is not easy to read back the design thinking behind them. As a result, the tendency is to apply local fixes only to the step where an error occurred, without considering the overall structure, and that is one of our significant operational challenges.

Ito: I see — the more complex the environment, the more important it becomes to triage before running tests. Being able to reduce unnecessary executions through early detection is a meaningful benefit. And beyond just fixing the error location, supporting developers in making corrections that account for the broader context — in a way that is simpler and more accurate — seems like it will become increasingly important going forward.

Kawase: Exactly. On that point of "support for making the fix," I feel that MagicPod Autopilot would become even more powerful if it were extended to support Shared Steps. Even now, I have confirmed that when data patterns are in use, Autopilot appropriately retrieves the relevant variables — and that seems like something we can put to use. For things like calculation logic that I don't work with often, it feels easy to delegate the finer details to it.

Ito: We talked about Cursor and MCP for impact assessment earlier, but there is a step beyond that worth mentioning as well. Depending on how it fits your operational flow — are you currently using the approach of reviewing code in an AI editor and then applying those review results directly to test fixes via MCP?

Kawase: We are not doing that yet.

Ito: Actually, it has recently become possible to call Autopilot from MCP, so that by instructing the editor to "apply these review results to the tests as well," new test creation and modifications can be carried out automatically. At this point in the current specifications, changes are applied directly to the main branch rather than a test branch, so there is one extra step needed — checking the execution history to confirm everything looks correct.

Kawase: For newly created tests, having changes applied directly to the main branch is completely fine. That sounds very useful — I will definitely try it. Thank you.

Recently, AI-powered code generation and similar tools have dramatically accelerated the speed of feature development, and there is a shared sense of urgency within the company that if QA remains reliant on traditional manual testing, it will become a bottleneck for the entire release process.

Ito: How to keep QA pace with development speed as AI continues to accelerate it is a major theme for the industry as a whole. What will be needed going forward is not disposable tests written purely for speed, but tests that are resilient to change and built for stable long-term operation. MagicPod will keep evolving to support that.

---

Closing

Kawase: In our initial meetings after implementation, you shared common failure patterns and operational tips with us upfront — and that enabled us to design a structure with long-term operation in mind from the very beginning. Looking back on these past two years, I am genuinely grateful that we got that initial foundation right.

I think most engineers today understand the value of automating regression testing. But the true value, I would argue, lies in being able to achieve that "without depending on any one individual's technical ability — with consistent quality regardless of who touches it." My honest assessment is that adopting MagicPod was the right call — as a tool for embedding an automation culture across the organization and eliminating over-reliance on individuals.

Hacobu Co., Ltd.

• Corporate site: https://hacobu.jp/
• Careers: https://career.hacobu.jp/
• Tech blog: https://zenn.dev/p/hacobu
• Tech entrance book: https://hacobu.notion.site/entrance-book-engineers

---

From left:
Takahiro Yamauchi, Product & Service Unit
Mitsuhiro Oishi, Product & Service Unit, General Manager
Nozomi Ito, MagicPod CEO

---

Mitsuhiro Oishi (hereafter Mitsuhiro): I joined the company in 2008, when the development team had about 10 engineers. In 2012, we launched PORTERS — a matching system specialized for staffing and recruitment, and have continued updating it ever since.

PORTERS is a SaaS platform that centrally supports the core operations of staffing businesses, from job order management and candidate management to hiring progress tracking and revenue management. Because we continuously expand our feature set to accommodate the industry's complex workflows, figuring out how to build an organization that balances development speed with quality has been my overarching challenge.

Testing was originally handled by the development engineers themselves. Around 2011, our CTO at the time joined and established a QA team, and as the service grew, our quality assurance structure gradually took shape. For test automation specifically, that has been Takahiro's domain.

Takahiro Yamauchi (hereafter Takahiro): I joined PORTERS as a contractor about two years ago. I've been involved in testing for around 20 years — as a QA chief, team lead, and sometimes as a tester. On top of that I had been driving automation at my previous company as well. However, the tool we introduced there would sometimes fail to run scenarios we had created, and other team members were struggling with the same issues, so I had developed the impression that automated testing was something that just didn't work reliably.

---

Challenges Before Implementing MagicPod

Takahiro: Actually, before joining PORTERS, I was involved in PORTERS' testing as a member of an external company. I wasn't directly testing the PORTERS service itself, but even then, looking at the sheer volume of features, I thought, "the people testing all of this must have it rough." When I actually joined as a contractor and got inside, it was exactly as I had imagined. (laughs)

Mitsuhiro: Our QA operations are currently handled by VNEXT, a Vietnamese offshore company. They manage everything from test case creation to manual test execution. But as PORTERS' feature set kept growing, the time required for testing became a serious problem at a certain point.

The moment we felt the most pressure was during middleware version upgrades. Version upgrades are essential for maintaining strong security, but when a full test cycle takes three months, it means no new releases can go out during that entire period. Our biggest concern was development coming to a halt when we wanted to be delivering more value to our users — we had reached a point where manual testing simply wasn't sustainable anymore.

On top of that, there were areas where a programming error could lead to information leaks — such as email sending and external data integrations. We had a strong desire to automate testing for those areas to bring regression defects as close to zero as possible.

Three or four years ago, our engineers had actually tried building automated tests using Selenium. With features expanding as the service grew and manual testing falling behind, everyone pitched in to write code and we made it through that period. But once the engineers returned to development work, there was no bandwidth left for maintenance, and the automation ultimately never became a sustainable operation.

---

---

Why and How MagicPod Was Selected

Nozomi Ito (hereafter Nozomi): With those challenges in mind, how did you go about selecting a tool?

Mitsuhiro: We compared about three options, and the deciding factors were the number of test executions and ease of implementation and operation.

On the features side, MagicPod has no limit on the number of test executions. Because we release frequently to deliver value to our customers as quickly as possible, and because we need to run tests with every security update, we determined that any cap on executions would make it impossible to keep up.

On the operations side, our experience with Selenium had taught us that we needed to avoid tools so complex that only a handful of people could use them. Knowing that MagicPod could realistically take root on the ground and be continuously maintained was another major reason for choosing it.

We completed the selection in about a month and have been using it since November 2023. We started by having the VNEXT team study MagicPod and building test cases for a subset of features such as email-related functionality — however, at that initial stage we weren't able to get it running smoothly. So we went looking for an external partner to help formalize our automated testing operations, reached out to about three companies, and ultimately brought in a third-party verification firm.(Note: The third-party verification firm's support was provided directly to PORTERS Co., Ltd.)

The firm had deep expertise in MagicPod, and what really made the difference was that they said they would thoroughly share their knowledge with the VNEXT team as well.

Takahiro: After the third-party verification firm came on board, I also started engaging with MagicPod in earnest. Watching their approach to knowledge sharing and maintenance, I found it completely different from the tools I had used before. It was easy to build in, easy to read, easy to verify. I thought, "this is really good."

---

---

Using MagicPod

Mitsuhiro: The third-party verification firm supported us for about six months, during which we built test cases starting from the most critical features. That structure has now found its footing, and the VNEXT team is able to operate autonomously. Takahiro manages things from the Japan side, with a VNEXT team of 13 people on manual testing and 4 on automated testing.

Because the manual and automated teams both sit within VNEXT, we're able to create a strong feedback loop — defects caught by automated tests get fed back into the manual test cases, and conversely, bugs found during manual testing of new features get incorporated into the automated test scenarios. That mutual integration has been hugely valuable.

Takahiro: Execution runs mainly overnight and during daytime hours on weekends, which are times when team members aren't working — running 2 instances in parallel. The number of test cases has grown considerably, so even with parallel execution it takes about three days to complete a full cycle. The QA team checks the results first, and if there are any new errors or scenarios that stopped, I get notified. I then review the results on my end as a double-check to make sure nothing was missed.

One thing that's particularly helpful from an operational standpoint is how much easier the test results are to read. Screenshots are captured at the point of failure, so you can immediately see what was being done, where, and what happened. I also find it very useful that when a test encounters an error, you can choose whether to stop there or continue running — that flexibility matters a lot in practice.

For example, if execution stops because of a minor error like a slight text difference, you have no way of knowing whether everything after that point is fine or not. Being able to continue running through an error and check all subsequent behavior in one go is a significant operational advantage.

Mitsuhiro: When we ran the numbers, we had roughly 30,000 test cases in total. Running all of them as manual tests cost approximately 6 million yen in labor per cycle. By incorporating automated testing, we've been able to bring that down to approximately 1 million yen — a reduction of roughly 5 million yen per full test run.

On top of that, the full test period has been cut from three months to approximately two weeks, which means development no longer has to stop during testing — and that impact is enormous. The system has also been reliably catching defects, detecting multiple regression issues, which has been a huge help.

Takahiro: A little while after I joined, I asked the manual testing team, "How long would it take to do a full test right now?" They said, "Six months." Now, with the exception of some areas like admin screens, we've reached a state where we can cover nearly all the features a standard user would interact with.

The execution environment is also flexible — simply changing the base URL lets us switch between production and staging, so we can check for regression defects in the customer-facing environment or verify that there are no regressions outside of new features in the development environment. Automated testing has also become well understood within the development team. We now regularly get messages asking, "How soon could you complete a full cycle if we requested it now?"

Mitsuhiro: There's been a change in terms of knowledge concentration as well. PORTERS has accumulated over a decade of features, and even current development team members regularly encounter things like, "I didn't know this feature existed." Automated testing covers those areas that tend to get overlooked, and whenever a regression occurs, we add a test case so the test suite grows over time. Having a system where, even as team members change, things like "this area slipped through the cracks" become less likely to happen. I think that's genuinely valuable.

Nozomi: Are there any practices or approaches you've developed to keep daily operations running smoothly?

Mitsuhiro: Actually, we've intentionally chosen not to use the notification feature. Every day, the VNEXT team members go directly to check the results themselves to see whether any errors have occurred. Automated testing will always have things that don't go perfectly, so rather than relying on notifications, having dedicated people doing daily checks is something we think is essential to keeping the operation running continuously.

Takahiro: There's also a specific approach we've developed for handling the quirks of automated testing. For example, in tests that verify email body content, MagicPod performs strict comparison down to invisible line break differences, which can cause failures. But in practice, there's no visible impact on the customer-facing screen. For defects that only appear in automated testing, rather than forcing changes to the scenario, we've adopted the practice of letting those run as known failures. We believe it's important not to get too caught up in errors that have no customer impact, and to make judgment calls with flexibility.

---

---

From "Automate for Now" to Sustainable Test Operations

Nozomi: You mentioned that there was a period early on after implementing MagicPod where things didn't get off the ground smoothly. What specifically made it difficult?

Mitsuhiro: At first, we tried to directly replace manual tests with automated ones. We were building in granular checks like "did this text change or not?" But what matters in automated testing isn't that — it's whether the features are functioning correctly.

When you're checking areas where minor text differences don't matter, execution becomes a burden and failures pile up to the point where it stops being useful. We handed things off without getting that sorted out, which is why we felt it would be difficult to scale.

On top of that, this was before Takahiro had joined, so we didn't have test design expertise to begin with. VNEXT wasn't in a position to lead in that area either, and we felt we'd hit the limits of what we could do on our own — that's what led us to reach out to a third-party verification firm. Once the firm came on board, they reviewed things from the ground up, starting with the fundamental question of how to approach building test cases, and that knowledge accumulated within the VNEXT team.

Takahiro: We ourselves didn't have deep expertise in MagicPod at that point either, so the guidance from the firm was genuinely educational. They taught us things on a near-daily basis — such as how to restructure test procedures to parallelize multiple test cases and shorten the execution period, and how to ensure test case updates are made reliably without anything slipping through.

Shortly before now, we expanded the automated testing team within VNEXT from 2 members to 4. The knowledge that the original 2 had gained from the firm was passed on smoothly to the new members, and they got up to speed without any significant issues. The fact that knowledge is now properly handed down is something I think is directly attributable to the hard work we put in during the initial setup period.

Nozomi: Were there any challenges on the product side in terms of element identification?

Mitsuhiro: PORTERS is built with React, so IDs are auto-
generated and can change, and users can freely configure screen fields, meaning what appears where can change day to day. But MagicPod handles this well — it's working without issues. We haven't had to ask the development team to make any special accommodations so far.

Takahiro: Occasionally, we get a message saying, "There's an element we can't capture, so we'll handle it with coordinate-based targeting," but it happens very rarely.

---

---

Closing

Takahiro: I believe that regardless of the system, the testing man-hours required for regression testing are never trivial. Automated testing is a highly effective solution for addressing that. Getting from zero to one is challenging, as we've discussed. But once you reach that point, it becomes an incredibly powerful asset. MagicPod in particular is easy to use and has excellent maintainability, so I can recommend it with confidence. It's been a huge help for us, and we hope to continue using it for a long time to come.

Mitsuhiro: We had our struggles with automated testing in the past, but now Takahiro manages it and VNEXT's dedicated team runs it every day. Being able to build this structure of "always keeping it running" has been truly significant. Automated testing will always produce errors, so the key is checking daily and keeping it in a consistently working state. That's what makes it actually succeed. I shudder to think about where we'd be without it now.

Being able to run tests as part of everyday processes rather than only before releases has allowed us to treat testing not as a special activity, but as an integral part of development. In the SaaS world, working software is a baseline expectation — if there are defects, customers simply can't use the product. Being able to ensure quality through testing and release with confidence is, I think, enormously valuable.

PORTERS Co., Ltd.

• Corporate site: https://www.porters.jp/
• PORTERS Tech Blog: https://zenn.dev/p/porters_tech

---

---

---

Counterclockwise from top left:
● Mai — Sales Team Leader
● Men — Test Leader
● Tam — Test Leader
● Nozomi Ito — MagicPod CEO

---

VNEXT: My name is Mai, and I lead the Sales team. Joining me today from the team that uses MagicPod are our Test Leaders, Tam and Men. Both of them can read and write Japanese, but speaking is a bit of a challenge, so I will be speaking on behalf of the group today.

Ito: Thank you. This is actually the first time we have had the chance to hear from a customer using MagicPod outside of Japan, and I have been looking forward to it. To start, could you give us a brief introduction to VNEXT?

VNEXT: We have been operating for 18 years since our founding in 2008, delivering high-quality, fast-turnaround development through an offshore model that combines resources in Japan and Vietnam. Today, our group totals approximately 565 people — 65 in Japan and around 500 in Vietnam — with a track record of over 400 clients and more than 800 projects completed.

One distinctive feature of our organization is that rather than having developers handle their own testing end-to-end, we maintain a dedicated, independent test team responsible for quality assurance. Our view is that when developers test their own code, the focus tends to stay on logical correctness, which can make it easy to miss things like subtle UI rendering issues.

We serve a broad range of industries; manufacturing, logistics, retail, services — handling everything from small-scale operational improvements to large-scale core system development. In recent years, we have also been investing heavily in AI, incorporating it into our own development processes to improve efficiency and optimize costs.

---

Challenges Before Implementing MagicPod

VNEXT: The biggest challenge in offshore development is ensuring that test quality does not depend on any one individual's skills or experience. As projects scaled, we needed a system that would allow tests to be executed to the same standard regardless of who was running them. When we were relying on manual testing, quality varied depending on who performed the work, and as a result, it was often difficult to give clients clear, evidence-based answers when they asked what justified our quality assessments.

Ito: So the motivation for exploring automation tools was to eliminate that knowledge dependency?

VNEXT: Exactly. And many of our testers on the Vietnam side are not able to write code. That made tools like Selenium or Appium — which require coding skills as a baseline — too costly to onboard, and rolling them out across the whole team was not realistic. When only a limited number of people can use a tool, the knowledge-silo problem simply resurfaces in a different form. On top of that, the complexity of environment setup and maintenance is a significant barrier in a remote, offshore context. What we needed was an automation framework that anyone could use and anyone could understand.

---

Why VNEXT Chose MagicPod

VNEXT: The trigger for full-scale adoption was a request from a Japanese client at the end of 2022, when test automation was included as a project requirement. In fact, we had already been researching MagicPod since around 2021 and had been using it internally on a trial basis — we had been looking into it ahead of time to prepare for recommending it to clients, so the timing aligned well.

In response to the client requirement, we conducted a fresh comparison of multiple tools. The three criteria we prioritized were: whether non-coders could use it, whether environment setup burden was minimal, and whether it could standardize quality management across the whole team. MagicPod came out on top, we proposed it to the client, received approval, and moved to full-scale deployment in April 2023.

Ito: There are a few other well-established automation tools out there — do you work with those as well?

VNEXT: We do recommend MagicPod to clients based on their needs, but depending on the project, another tool may end up being selected. The split is roughly fifty-fifty. The team currently using MagicPod has about 20 people.

Ito: What was your first impression once you actually started using it?

VNEXT: The first thing I would highlight is how easy it is to maintain. In test automation, the long-term cost of upkeep often becomes a bigger challenge than the initial test case creation — but with MagicPod, you are unlikely to end up in a situation where automation becomes self-defeating because maintenance consumes all your time.

There is always some turnover on the Vietnam team, but the fact that new members can quickly understand and modify tests built by their predecessors is enormously valuable in day-to-day operations.

---

How VNEXT Uses MagicPod

Ito: How do you train new members to get up and running with MagicPod?

VNEXT: MagicPod is not particularly difficult to learn. For someone who already has manual testing experience, one to two weeks of hands-on training is enough to get them productive. With automation tools that require code, it can take anywhere from six months to a year to reach proficiency, but MagicPod is no-code and intuitive to operate. The time it takes for new members to understand and start modifying test cases has been dramatically reduced compared to our previous Selenium environment.

Ito: What kinds of results have you seen in actual operations?

VNEXT: We are primarily using MagicPod for E2E test automation across web applications and mobile apps, with regression testing integrated into the CI/CD pipeline and running on a nightly schedule. This has increased our testing frequency from once or twice a week, leading to earlier defect detection, and has cut the testing period per release from two to three days down to under half a day.

We have two teams: a manual testing team and an automation testing team — and the automation team's effective use of MagicPod has indirectly boosted the manual team's productivity and reduced their man-hours as well. For example, work that previously took the manual team 10 hours now takes around 8 hours, a roughly 20% reduction. The freed-up capacity is being reinvested in improving test design quality and covering the next set of features.

For our scheduled execution, a workflow that makes good use of the two-hour time difference between Japan and Vietnam has become standard practice. If the Vietnam team sets the tests running at the end of their workday, all results are ready by the next morning. MagicPod has a setting that allows tests to run to completion even when errors are encountered, rather than stopping mid-way, which has been a real help.

This means the Vietnam team can review errors and defects in advance, and be ready to report and consult with the Japanese client smoothly as soon as they arrive at the office. In a remote environment, we have effectively achieved around-the-clock QA coverage without any additional cost.

Ito: You have really built an efficient cycle by making good use of the time difference. On the other hand, working across Japan and Vietnam must come with some infrastructure challenges — did you experience any friction around remote environment setup or access?

VNEXT: None at all. Because MagicPod is cloud-based, everything runs in the browser without any dependency on VPNs or remote desktops. Both Vietnam and Japan access the same environment, so there are no issues caused by environment discrepancies. When a new member joins, all it takes is creating an account — they can start working in the same environment immediately. That is a significant advantage in an offshore setting.

Ito: Now that you have eliminated those environment discrepancies, has anything changed in how you communicate with your Japanese clients?

VNEXT: The biggest change is that test results have become something we can actually show. In a traditional remote environment, the only option was to report test progress in text, and it was often difficult to convey precisely to clients what had been checked and to what extent.

With MagicPod, each step's execution screen is automatically saved as a screenshot, so the test results themselves become the audit trail. Instead of asking clients to "read a report," the dynamic has shifted to "reviewing results together."

Follow-up questions about test reports have become almost nonexistent, and the time we used to spend on explanations can now go toward more meaningful discussions about quality improvement. Our clients have spoken highly of the team, and we believe MagicPod is one of the key factors behind that.

Ito: I am really glad to hear it is contributing to quality improvements. I understand that one aspect that works particularly well in your offshore context is the fact that MagicPod's Help Center is available in English. How has that been?

VNEXT: Our test engineers are primarily people who can communicate in Japanese for business purposes, but when issues arise, we are seeing more cases where engineers can resolve them directly by consulting the English Help documentation, without needing to go through a Japanese-speaking intermediary. That has reduced communication bottlenecks and improved our response speed.

---

What the Offshore Team Expects: AI-Powered Acceleration of Test Design

Ito: I understand there is also interest from your team in MagicPod Autopilot — how much are you currently using it in practice?

VNEXT: On our team's side, we are still in the research phase — it is mainly our clients who are using that feature at the moment. That said, we see enormous potential in it as an offshore team. The ability to create and edit test cases in natural language is something we want to actively explore and expand our use of.

We currently run tests in Japanese, but if we could simply give instructions in English and have the steps generated automatically, we expect test design speed to increase significantly. Our teams on the ground are genuinely excited to see how this can help solve the challenge of leveraging generative AI to improve productivity.

May I ask a question? Is MagicPod's development currently handled mainly by engineers on the Japan side?

Ito: Yes, all development is done in-house in Japan. We have around 20 engineers, but recently we have been seeing more internationally-sourced members joining; people living in Japan who are originally from the US, India, Hong Kong, and elsewhere.

VNEXT: I see. So alongside expanding the tool globally, you are also building out a globally diverse team regardless of nationality. That is impressive.

Ito: Yes, gradually but steadily; we want to grow both our hiring and our sales on a global scale.

---

Closing

VNEXT: Implementing a test automation tool tends to be framed as an engineering challenge, but in practice it is also an organizational challenge — a question of how to get the whole team moving together. No matter how good a tool is, if only certain members can use it, you end up right back at the same knowledge-dependency problem you started with.

The biggest reason we are genuinely glad we chose MagicPod is that the entire team can now approach testing with the same tool and the same standard. Being no-code is not just about being "easy to use" — it means being "usable by everyone." In a team with the diverse backgrounds typical of an offshore environment, that distinction carries enormous weight.

Test automation does not deliver its value the moment you implement it. It has to take root in the team and keep running every day before the real impact becomes visible. MagicPod has firmly met our expectations as a tool that supports that process of embedding automation into how a team actually works. Our experience is that starting small, building confidence, and expanding from there is the surest path to success.

VNEXT HOLDINGS JOINT STOCK COMPANY

• Corporate website: https://vnext.co.jp/
• V-BLOG: https://vnext.co.jp/v-blog.html

---