1、 Video Special Effects Peng Huang 2.1.2 Object-space NPAR Meier was the first one who produced painterly animations from object-space scenes53.He triangulated surfaces in object-space and distributed strokes over each triangle in proportion to its area. Since his initial work, many object-space NPAR
2、 systems have been presented. Halls Q-maps29(A Q-map is a 3D texture which adapts to the intensity of light to give the object in the image a 3D look, for example, more marks are made where an object is darker) may be applied to create coherent pen-and-ink shaded animations.A system capable of rende
3、ring object-space eometries in a sketchy style was outlined by Curtis15, and operates by tracing the paths of particles traveling stochastically around contours of a depth image generated from a 3D object. See Figure 2.3 for some examples.In addition, most modern graphical modeling packages(3D Studi
4、o MAX!,Maya,XSI Soft-Image) support plug-ins which offer the option of rendering object-space scenes to give a flat shaded, cartoon-like appearance. 2.1.3 Image-space NPAR Most NPAR systems in image-space are still based on static painterly rendering techniques,brushing strokes frame by frame and tr
5、ying to avoid unappealing swimming which distractsthe audience from the content of the animation. Liwinowicz extends his static method and makes use of optical flow to estimate a motion vector field to translate the strokes painted on the first frame to successive frames47. A similar method is emplo
6、yed by Kovacs and Sziranyi42. A simpler solution is proposed by Hertzmann33, who differences consecutive frames of video, re-painting only those areas which have changed above some global(userdefined) threshold. Hays and Essas approach32 builds on and improves these techniques by using edges to guid
7、e painterly refinement. See Figure 2.4 for some examples. In their current work, they are looking into studying region-based methods to extend beyond pixels to cell-based renderings, which implies the trend from low-level analysis to higher-level scene understanding. We also find various image-space
8、 tools which are highly interactive to assist users in the process of creating digital non-photorealistic animations. Fekete et al. describe a system23 to assist in the creation of line art cartoons. Agarwala proposes an interactive system2 that allows children and others untrained in “cel animation
9、” to create 2D cartoons from images and video. Users have to hand-segment the first image, and active contours(snakes) are used to track the segmentation boundaries from frame to frame. It is labor intensive(usersneed to correct the contours every frame), unstable(due to susceptibility of snakes to
10、local minima and tracking fails under occlusion) and limited to video material with distinct objects and well-defined edges. Another technique is called “advanced rotoscoping” by the Computer Graphics community, which requires artists to draw a shape in key-frames, and then interpolate the shape ove
11、r the interval between key-frames a process referred to as “in-betweening” by animators. The film “Waking Life”26 used this technique. See Figure 2.5 for some examples. NPAR techniques in image-space as well as commercial video effects software, such as Adobe Premier, which provide low-level effects
12、(i.e slow-motion, spatial warping, and motion blur etc.), fail to do a high-level video analysis and are unable to create more complicated visual effects(e.g. motion emphasis). Lake et al. present techniques for emphasizing motion of cartoon objects by introducing geometry into the cartoon scene43.
13、However, their work is limited to object-space, avoiding the complex high-level video analysis, and their “motion lines” are quite simple. In their current work, they are trying to integrate other traditional cartoon effects into their system. Collomosse and Hall first introduce high-level Computer
14、Vision analysis to NPAR in “VideoPaintbox”12. They argue that comprehensive video analysis should be the first step in the artistic rendering(AR) process； salient information(such as object boundaries or trajectories) must be extracted prior to representation in an artistic style. By developing nove
15、l Computer Vision techniques for AR, they are able to emphasize motion using traditional animation cues44 such as streak-lines, anticipation and deformation. Their unique contribution is to build a video based NPR system which can process over an extended period of time rather than on a per frame ba
16、sis. This advance allows them to analyze trajectories, make decisions regarding occlusions and collisions and do motion emphasis. In this work we will also regard video as a whole rather than the sum of individual frames. However, their segmentation in “computer vision component” suffers labor inten
17、sity, since users have to manually identify polygons, which are “shrink wrapped” to the features edge contour using snake relaxation72 before tracking. And their tracking is based on the assumption that contour motion may be modeled by a linear conformal affine transform(LCAT) in the image plane. We
18、 try to use a more automatic segmentation and non-rigid region tracking to improve the capability of video analysis. See Figure 2.6 for some examples. Another high-level video-based NPAR system is provided by Wang et al.69. They regard video as a 3D lattice(x,y,t) and then implement spatio-temporal
19、segmentation of homogeneous regions using mean shift14 or improved mean shift70 to get volumes of contiguous pixels with similar colour. Users have to define salient regions by manually sketching on key-frames and the system thus automatically generates salient regions per frame. This naturally buil
20、d the correspondence between successive frames, avoiding non-rigid region tracking. Their rendering is based on mean shift guided interpolation. The rendering style is limited to several approaches, such as changing segment colour and placing strokes and ignores motion analysis and motion emphasis.
21、Our system segments key-frame using 2D mean shift, identifies salient regions and then tracks them over the whole sequence. We extract motion information from the results and then do motion emphasis. See Figure 2.6 for some examples. 2.1.4 Some other NPAR techniques Bregler et la present a technique
22、 called “cartoon capture and retargeting” in 7 which is used to track the motion from traditional animated cartoon and then retarget it onto different output media, including 2D cartoon animation, 3D CG models, and photo-realistic output. They describe vision-based tracking techniques and new modeli
23、ng techniques. Our research tries to borrow this idea to extract motion information, from general video rather than a cartoon, using different computer vision algorithms and then represent this in different output media. See Figure 2.7 for some examples. 2.1.5 NPR application on Sports Analysis Due
24、to a more and more competitive sports market, sports media companies try to attract audiences by increasingly providing more special or more specific graphics and effects. Sports statistics are often graphically presented on TV during sporting events such as the percentage of time in a soccer game t
25、hat the ball has been in one half compared to the other half. These statistics are collected in many ways both manually and automatically. It is desirable to be able to generate many statistics directly from the video of the game. There are many products in the broadcast environment that provide thi
26、s capability. The Telestrator78 is a simple but efficient tool to allow users to draw lines within a 2D video image by using a mouse. The product is sold as a dedicated box with a touch screen, a video input and video outputs the video with the graphics produced. Typically, four very simple controls
27、 such as draw arrow, draw dotted line etc. are provided. 视频特技 黄鹏 2.1.2 三维空间的 NPAR Meier 第一 个 在绘画上创造出三维感觉 ，他把物体按比例 绘制到三 维界 面 上实现在二维介质上实现立体感， 有了他的理论工作，许多三维空间的 NPAR开始被研发出来。 Hall的 q-maps（ q-maps是 一种利用光影制造出三维感觉的 质感， 例如， 很多物体都有黑色阴影） 可以制作出 互相密合着的钢笔画 般 的黑白动画。Curtis提出一种利用几何学有力表现立体空间的系统 ，而且根据追踪操作在附近随机程序旅行的粒子的
28、路径深度图像的等高线从而产生 3D立体物体。 如图例 2.3 除此之外，大部分的图形工作界面软件（ 3D Studio MAX!,Maya,XSI Soft-Image）支持图形操作带来的三维立体感的插件。 2.1.3二维空间的 NPAR 大部分的 NPAR二维系统仍然使用静图显示技术， 一种用画笔一帧一帧的描绘还要避免丢帧带来的跳动感 。 Liwinowicz发展了这种静图描绘方法 ，利用光学流程到估计运动矢量领域呈现笔划着色的第一个帧上对应连续的帧 47。Kovacs 和 Sziranyi 使用了类似的方法。 Hertzmann33提出了一种简便的解决方法， 他区分出不同连续的帧，只描绘出有动作变化的局部而不是全部改变 。 Hays 和 Essa的方法 通过 精致的描绘边缘 改进 了 技术 。 看图例 2.4 。 在他们最近的研究中 ，他们研究局部为主的方法来改进 以 像素 为单位 显示 ，从而暗示 他们对 那些由低到高分辨显示的理解。
