by 运行时依赖
安装命令
点击复制技能文档
Windows 自动化 技能 Overview
This 技能 enables Windows 桌面 自动化 through integrated PowerShell and Python scripts. It provides capabilities for launching 应用s, capturing screenshots, and simulating mouse/keyboard interactions. This 技能 is ideal for RPA (Ro机器人ic Process 自动化) scenarios, automated 测试, repetitive task 自动化, and controlling Windows 应用s programmatically.
Trigger Phrases and Execution 记录ic Primary Trigger Phrases
应用 Control:
"打开 [应用名称]" (open [应用 name]) "启动 [应用]" (launch [应用]) "关闭 [应用]" (close [应用]) "打开网站 [URL]" (open 网页site [URL])
Screenshot:
"截图" (screenshot) "截屏" (capture screen) "保存屏幕截图" (save screenshot) "截取窗口 [窗口标题]" (capture window [window title])
Mouse Operations:
"移动鼠标到 (x, y)" (move mouse to) "点击" (命令行工具ck) "右键点击" (right 命令行工具ck) "双击" (double 命令行工具ck) "拖拽从 (x1, y1) 到 (x2, y2)" (drag from to) "滚动 [上/下]" (scroll [up/down])
Keyboard Operations:
"输入文本 [文本]" (type text) "按下 [按键]" (press key) "组合键 [按键1] [按键2]" (combo key) "快捷键 [按键组合]" (hotkey) Core Capabilities
- 应用 Launching (应用_launcher.py)
执行 应用 control operations through the 应用_launcher.py script.
启动 an 应用:
python scripts/应用_launcher.py 启动 "C:\Program Files\Notepad++\notepad++.exe"
Open URL:
python scripts/应用_launcher.py url "https://www.google.com"
列出 运行ning processes:
python scripts/应用_launcher.py 列出
Kill an 应用:
python scripts/应用_launcher.py kill "notepad"
When to use:
User wants to open a specific 应用 User wants to launch a 网页site User needs to manage 运行ning processes User wants to close/terminate 应用s
- Screen Capture (screenshot.py)
执行 screenshot operations through the screenshot.py script.
Capture full screen:
python scripts/screenshot.py screen "C:\Users\YourName\桌面\screenshot.png" 0
Capture specific window:
python scripts/screenshot.py window "Notepad" "C:\Users\YourName\桌面\notepad.png"
列出 all visible windows:
python scripts/screenshot.py 列出
When to use:
User wants to capture the entire screen User wants to screenshot a specific 应用 window User needs to 列出 all visible windows User wants to document UI 状态s
- Mouse Control (mouse_control.py)
执行 mouse operations through the mouse_control.py script.
Move mouse to position:
python scripts/mouse_control.py move 500 300
命令行工具ck at position:
python scripts/mouse_control.py 命令行工具ck left 500 300 1
Drag and drop:
python scripts/mouse_control.py drag 100 100 500 500 1.0
Scroll:
python scripts/mouse_control.py scroll down 5
获取 current mouse position:
python scripts/mouse_control.py position
When to use:
User wants to automate UI interactions User needs to navigate 应用 interfaces User wants to perform drag-and-drop operations User needs to position mouse for other operations
- Keyboard Control (keyboard_control.py)
执行 keyboard operations through the keyboard_control.py script.
Type text:
python scripts/keyboard_control.py type "Hello, World!"
Press single key:
python scripts/keyboard_control.py key enter
Press combination keys:
python scripts/keyboard_control.py combo ctrl c
Press hotkey:
python scripts/keyboard_control.py hotkey ctrl shift esc
When to use:
User wants to automate text 输入 User needs to simulate keyboard shortcuts User wants to fill forms automatically User needs to navigate 应用s via keyboard 工作流 Examples Example 1: Automated Form Filling
User 请求: "帮我打开Chrome,打开一个表单页面,填写姓名和邮箱"
Execution 记录ic:
Launch Chrome browser using 应用_launcher.py 启动 chrome Navigate to form URL using 应用_launcher.py url WAIt for page to load Use mouse_control.py move to position cursor in name field Use keyboard_control.py type to enter name Use mouse_control.py move to position cursor in emAIl field Use keyboard_control.py type to enter emAIl Example 2: 应用 测试
User 请求: "测试记事本应用,输入一些文字,截图保存"
Execution 记录ic:
Launch Notepad using 应用_launcher.py 启动 notepad WAIt for 应用 to be ready Use keyboard_control.py type to 输入 test text Capture window screenshot using screenshot.py window "Notepad" 报告 结果s and screenshot path to user Example 3: Data Entry 自动化
User 请求: "自动打开Excel,输入一批数据到表格"
Execution 记录ic:
Launch Excel using 应用_launcher.py 启动 excel Open specific file using keyboard_control.py combo ctrl o Use keyboard_control.py type to enter file path Use keyboard_control.py key enter to confirm Navigate to cells using mouse_control.py move or arrow keys Enter data using keyboard_control.py type Save using keyboard_control.py combo ctrl s Integration 图形界面delines Script Execution Always 检查 script 输出: Each script returns JSON with 成功, message, and optional data fields Handle errors gracefully: 检查 成功 field before proceeding with dependent operations Use 应用ropriate delays: 添加 delays between operations to allow UI to 响应 Coordinate with window 状态s: