Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,86 @@ With any luck, you now have a functioning version of Gaffer.
../gaffer-build/bin/gaffer
```

### Build process on Windows 11###

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a space between the 11 and the trailing # signs. Otherwise it will display the actual hash symbols rather than interpret it as a formatting code.


#### 1- Use Git Bash

```bash
# Must be executed inside Git Bash to interpret the .sh file
./config/installDependencies.sh ../gaffer-build

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the use of installDependencies.sh (also mentioned above in the main build instructions) is out of date now. Even easier than using git bash is to run .github/workflows/main/installDependencies.py. On Linux you can run that script directly, on Windows you need to do python .github/workflows/main/installDependencies.py (Windows doesn't recognize Python scripts as being executable).

Might be worth putting this into the shared build steps since it's now cross-platform?

```
Once the dependencies finish unpacking into your gaffer-build directory, close Git Bash.

#### 2- Install Scons

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kind of inclined to let this step be covered by the Main build requirements section above, where scons is mentioned. I don't think scons is installed by default on most Linux distributions, so maybe it's worth adding a note to the shared build steps that it can be installed using pip install scons?


```powershell
:: 1. Navigate to your local clone directory
cd path\to\your\gaffer

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this shouldn't be needed since pip will install scons for the user / system-wide Python interpreter, independent of the current directory.


:: 2. Install SCons into your python environment if you haven't already
pip install scons
```

#### 3- Add existing 7-Zip to your System PATH

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! This is definitely missing from the instructions and could trip up someone using the installDependencies.py script. I think it might be helpful to add it to the Main build requirements a bit further up in this file.

I'm wary though of including instructions for adding it to the system PATH. That requires administrator privileges that I'm guessing users in a lot of studios won't be able to elevate to.

I think the 7zip installer should handle adding itself to PATH, did you find that not to be the case?

- Typically, it is located at C:\Program Files\7-Zip. Check if 7z.exe is sitting inside that folder.
- In your Windows Search bar, type "environment variables" and select Edit the system environment variables.
- Click the Environment Variables... button at the bottom right.
- In the System variables list (the bottom section), find Path and click Edit....
- Click New and paste the path: C:\Program Files\7-Zip
- Click OK on all windows to save.
- Restart your terminal and run the script again.

#### 4- Force Python to use UTF-8 encoding in this PowerShell window

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch here too, this is needed and unique to Windows. Similar to above, I'm a little reluctant to add it to the system variables. I assume it requires elevation to admin, or the Powershell session itself needs to be admin? It would also have side effects for other Python sessions that may cause issues.

In Powershell, you can do $env:PYTHONUTF8=1 to set it for just the terminal session you're in and not have to worry about it affecting other uses of Python.


```powershell
[System.Environment]::SetEnvironmentVariable("PYTHONUTF8", "1", "User")
```

#### 5- Scons tips for laters

Before you can build you need to setup a few things because of the difference between linux and windows. Windows requires a little bit more hand holding. Make sure you do these steps below for a successful and fluid build.

***1. Inkscape Errors:*** Gaffer utilizes Inkscape purely during the compilation phase to automatically convert all of the UI icon assets (.svg vector files) into .png raster graphics for the application layouts. But the problem is that even though Inkscape might be in the environment path scons will not find it. So we have to force it in the command line. The line to add at the end of the build command is `INKSCAPE="C:\Program Files\Inkscape\bin\inkscape.exe" --config=force`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, which version of Inkscape are you using? I think this may be a problem with newer versions and holding back a few versions older may solve the problem.

We're using 1.2.2 in the CI builds (see .github/workflows/main.yml). I've been remiss about adding this to the version information for Inkscape in the section above.


***2. Scons cannot see installed git:*** Same issue with Inkscape we'll need to create a symbolic link to the real git for scons to see. Open power shell in elviated privilages

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is interesting, I wonder if this is something that's a problem in the Scons environment. Are you able to run git in the terminal you're building from, something like git -v to print version information? Is the git binary directory in your PATH? (You can do echo $env:PATH in your Powershell terminal to check it).

```powershell
# Create a link from your real git executable straight into the gaffer-build folder
New-Item -ItemType SymbolicLink -Path "C:\src\gaffer-build\bin\git.exe" -Target "C:\Program Files\Git\bin\git.exe"
```

***3. GLEW.lib:*** The linker will blow up looking for GLEW.lib, the fix is to find the real file, and create a mirror copy named exactly what the linker wants.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this, and possibly some of the Inkscape stuff above, can be helped by using the sconsOptions file. In the CI build, we add OPTIONS=.github/workflows/main/sconsOptions to the command to set some Scons options.

The key one in that options file for GLEW is GLEW_LIB_SUFFIX = "32" That should point your build to the right version and not need to horse around with hard-linking.

Open a normal PowerShell window and search for the actual GLEW library file inside your dependencies directory:
```powershell
Get-ChildItem -Path "C:\src\gaffer-build\lib\" -Filter "*glew*"
```
*(You will likely see `glew32.lib` or `glew32s.lib` print out in the list).*

Now, Open **PowerShell as an Administrator**, and create a hard link pointing from their expected name to the file that *actually* exists.
**If the file found was `glew32.lib`, run:**
```powershell
New-Item -ItemType HardLink -Path "C:\src\gaffer-build\lib\GLEW.lib" -Value "C:\src\gaffer-build\lib\glew32.lib"
```

***4. Build Time:*** By default, SCons acts conservatively and compiles everything on a single CPU thread. You need to explicitly tell it to spin up parallel workers using the -j (jobs) flag. ***Note: switching to multi core build will error out when reaching the Inkscape stage. Because you split the build into multiple parallel threads using `-j 8`, Gaffer launched 8 separate instances of Inkscape simultaneously, all trying to access the exact same SVG file at once. Inkscape on Windows often handles parallel thread calling very poorly, resulting in memory overflows and crashes. Since you only need to build these icons once (and they don't change when you tweak your RenderMan C++ node code), you can bypass this multi-threaded bug easily.Because SCons tracks file generation states, it will not recompile any of your C++ code. All your RenderMan and Gaffer UI objects are already safely compiled as .obj and .dll binaries from your last run. So make sure to run the build again with `-j 1` when the build errors out on the icons

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully this can be simplified by using the sconsOptions file I mention above, along with a better-working Inkscape. That options file will also take care of setting up RENDERMAN_ROOT, ARNOLD_ROOT, and DELIGHT_ROOT.

That options file does require having 3Delight and ONNX installed and configured, which can be done with Python scripts in the .github/workflows/main directory.


#### 5- Building on windows
Once all the tips in step #5 are completed open a fresh powershell and run the following command to build for various render engines

1. Default build with cycles: `scons build -j 16 BUILD_DIR=../gaffer-build NKSCAPE="C:\Program Files\Inkscape\bin\inkscape.exe"`
2. Build with RenderMan: `scons build -j 16 RENDERMAN_ROOT="C:\Program Files\Pixar\RenderManProServer-27.3" build BUILD_DIR=../gaffer-build NKSCAPE="C:\Program Files\Inkscape\bin\inkscape.exe"`
3. Build with Arnold: `scons build -j 16 ARNOLD_ROOT="C:\src\Arnold-7.5.2.0" build BUILD_DIR=../gaffer-build NKSCAPE="C:\Program Files\Inkscape\bin\inkscape.exe"`
4. Build with 3Delight: `scons build -j 16 DELIGHT_ROOT="C:\Program Files\3Delight" build BUILD_DIR=../gaffer-build NKSCAPE="C:\Program Files\Inkscape\bin\inkscape.exe"`

If you want to build for all engines just add reach location one after the other as following:
```powershell
scons build -j 16 RENDERMAN_ROOT="C:\Program Files\Pixar\RenderManProServer-27.3" ARNOLD_ROOT="C:\src\Arnold-7.5.2.0" DELIGHT_ROOT="C:\Program Files\3Delight" BUILD_DIR=..\gaffer-build INKSCAPE="C:\Program Files\Inkscape\bin\inkscape.exe"
```

***Once you hit the Inkscape error, redo the above command with***
```powershell
scons build -j 1 RENDERMAN_ROOT="C:\Program Files\Pixar\RenderManProServer-27.3" ARNOLD_ROOT="C:\src\Arnold-7.5.2.0" DELIGHT_ROOT="C:\Program Files\3Delight" BUILD_DIR=..\gaffer-build INKSCAPE="C:\Program Files\Inkscape\bin\inkscape.exe"
```

### Building with third-party renderer support

Gaffer dependencies ships with Cycles, but to build the modules for one of the other supported third-party renderers, you will need to set appropriate `scons` options pointing to your installation. The options are:
Expand Down
65 changes: 65 additions & 0 deletions startup/GafferRenderManUI/shaderMetadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -511,6 +511,71 @@

},

"osl:shader:PxrStylizedControl" : {

"noduleLayout:defaultVisibility" : False,

"parameters" : {

k : { "noduleLayout:visible" : True }
for k in [ "inputRGB", "Artistic_Light_Rotation_Vector", "lineAlbedo", "distortU", "distortV", "inputTextureCoords", "resultRGB", "resultAOV" ]

},

},

"osl:shader:PxrStylizedHatchControl" : {

"noduleLayout:defaultVisibility" : False,

"parameters" : {

k : { "noduleLayout:visible" : True }
for k in [ "Hatching_Signal_Input", "Input_RGB", "NPRalbedo2", "NPRalbedo3", "Input_Texture_Coords", "resultRGB", "resultAOV" ]

},

},

"osl:shader:PxrStylizedLightControl" : {

"noduleLayout:defaultVisibility" : False,

"parameters" : {

k : { "noduleLayout:visible" : True }
for k in [ "Input_RGB", "Artistic_Light_Rotation_Vector", "Result_RGB", "Result_Float", "Result_Shadow_Float", "Result_Lighting_Float", "resultAOV" ]

},

},

"osl:shader:PxrStylizedLinesControl" : {

"noduleLayout:defaultVisibility" : False,

"parameters" : {

k : { "noduleLayout:visible" : True }
for k in [ "Line_Color_RGB", "Normal_Map", "resultRGB", "resultAOV" ]

},

},

"osl:shader:PxrStylizedToonControl" : {

"noduleLayout:defaultVisibility" : False,

"parameters" : {

k : { "noduleLayout:visible" : True }
for k in [ "Input_RGB", "Signal_Input_Float", "Shadow_Input_Float", "resultRGB", "resultAOV" ]

},

},

}

for shader, metadata in shaderMetadata.items() :
Expand Down
Loading